Google Indexing And Mod_Rewrite

Posted by Robert on the 15th of January, 2005 at 1:00 AM GMT0. Permalink.
Tags: Geek, GoogleBot, mod_rewrite

The following is an advertisement.

When I posted this, I was listening to: The Connells - 74' - 75'

I've been having problems with Google's index not updating fast enough. If you search for something that was on the front page yesterday, and I've posted something new, you don't see it ('cause it is somewhere else).

So, I'm going to experiment. Google is only updating their public page rank every quarter now (supposedly), so it can't hurt if I try this for a month. Plus, I think my rank is up since I have some content (search on google for the phrase: Lindsey Lohan's New Album).

I added a googlebot meta tag to noindex,follow on any page that doesn't have a thread id or id in the querystring. Hopefully, that will force google to only index actual posts instead of the front page, and people's searches will turn up something real.

I also found out about mod_rewrite, which is part of Apache that allows .htaccess files to use regular expressions on the URL to move the user to the right page. We're using it on our new portal system at work to redirect a 404 file to a real address. Now, I just have to figure out how to get it to force Tammy Bellisle, Actress to the correct address instead of a subdirectory of my site.

But it's really a cool thing.

Add this page to del.icio.us or email it.

Comments:

1. Untitled

   by jes5199 on the 15th of January, 2005 at 11:11 PM GMT0. Permalink.

   Still not having a password that works.

   do you reply to my comments? i've never checked.

   anyway, mod_rewrite is the most fundamentally awesome thing there is. at work i use the windows equivalent: isapi_rewrite the entire site is based on that: all links look like /section_Whatever/134/ but are really just invocations of index.php

   even more impressive is the /bugtracker/ directory: it forwards all urls to magicforward.php , which loads the url on a different server entirely and displays the results to the user, without them even seeing it. it even passes POST and COOKIES transparently

2. Untitled

   by jes5199 on the 15th of January, 2005 at 11:15 PM GMT0. Permalink.

   woo, now i'm logged in

3. Untitled

   by Robert on the 15th of January, 2005 at 11:50 PM GMT0. Permalink.

   Damn. the slash-for-italics format is kinda fucking up directory listings. Wonder if it is worth having?

   As far as comments reply alerts, I'm going to do something about that eventually, but I need to do the redesign first. This design isn't going to handle some of the extra stuff I want.

   I'm still kinda perplexed by mod_rewrite. But the quote on the page sums it up perfectly:

   Despite the tons of examples and docs, mod_rewrite is voodoo. Damned cool voodoo, but still voodoo.

4. Untitled

   by jes5199 on the 16th of January, 2005 at 5:15 PM GMT0. Permalink.

   oh right, it requires that you already understand regexp.

   once you master that, it no longer looks like voodoo at all

5. Untitled

   by Robert on the 16th of January, 2005 at 5:38 PM GMT0. Permalink.

   My only problem is that you can't see the output for different variables to see what you are matching. The website doesn't really give you a lot of examples of what each one looks like. I always get errors about not being to access / and no explanation about what path I'm actually trying to get to (let alone the one I think I'm trying to get to).

6. Untitled

   by jes5199 on the 16th of January, 2005 at 7:00 PM GMT0. Permalink.

   i have no idea what you are talking about. provide examples?

7. Example

   by Robert on the 17th of January, 2005 at 11:21 AM GMT0. Permalink.

   I've got robertdot.org as my main domain on delta. I've also got tammybellisle.com pointed at the same account. I put Tammy's site in tammy/ and my site uses the root directory. On my old CrystalTech account, Windows ignored the domain name, so I could point tammybellisle.com/tammy and it would be fine. On delta, if you go to tammybellisle.com/tammy, it changes to robertdot.org/tammy.

   I used the following .htaccess mod_rewrite stuff to produce a you don't have access / on this server

   RewriteEngine on RewriteCond %{REQUEST_URI} ^.*tammybellisle.com.*$ RewriteRule ^(.*)/$ $1/tammy/

8. Untitled

   by jes5199 on the 18th of January, 2005 at 3:16 PM GMT0. Permalink.
   RewriteRule ^(.*)/$ $1/tammy/

   i'm pretty sure that this doesn't behave like you imagine it should you're appending all urls that end in slash with "tammy/"

9. Untitled

   by Robert on the 18th of January, 2005 at 7:09 PM GMT0. Permalink.

   If the address is tammybellisle.com, which is what the RewriteCondition is for, assuming I didn't read it wrong.

   But, I agree, delta isn't really set up to handle 2+ domains. Unfortunately, I don't want to pay an extra 30 something for another host and have to deal with other logins / passwords. But I might give in eventually.

10. Untitled

    by jes5199 on the 18th of January, 2005 at 4:31 PM GMT0. Permalink.

    mostly, Delta isn't set up to host two different domains

Comments on this page are closed due to age. If you need to say something, e-mail me.