Comments on: TTL and propagation http://theinnerlayer.softlayer.com/2007/ttl-and-propagation/ Do it faster. Do it better. Do it in private -- blog style. Thu, 04 Dec 2008 21:49:40 +0000 http://wordpress.org/?v=2.6 By: onegallon http://theinnerlayer.softlayer.com/2007/ttl-and-propagation/#comment-205 onegallon Tue, 20 Nov 2007 14:31:03 +0000 http://theinnerlayer.softlayer.com/2007/ttl-and-propagation/#comment-205 For the least possible amount of downtime, setting the TTL extremely low is a great first step, but here's a near-instant way of getting a new server operational with little user impact. (And it makes an allowance for DNS admins that ignore the TTL). 1. Lower the TTL on all DNS involved to an extremely low number. 2. Register a temporary secondary domain (i.e. www2), point it at the new server. 3. Set up the new server as if it were an exact copy. Test, test, test. 4. Make the switch. Flip on the new server, change IPs in the DNS. 5. Pull a final db copy from the old server to the new one for forums etc, shut down SQL on the old server to prevent orphans. 6. On the old server, enable a 307 redirect to the www2 URL. This way, while DNS propagates around the net, your users are getting redirected to the new server. 307 is Google-safe, and it also instructs web browsers to re-post any form data to the new URL. In a few days, traffic to the old server will taper off to nothing and you can take it offline. The only db-driven content downtime is the amount of time it takes to transfer the db and reattach it. For the least possible amount of downtime, setting the TTL extremely low is a great first step, but here’s a near-instant way of getting a new server operational with little user impact. (And it makes an allowance for DNS admins that ignore the TTL).

1. Lower the TTL on all DNS involved to an extremely low number.
2. Register a temporary secondary domain (i.e. www2), point it at the new server.
3. Set up the new server as if it were an exact copy. Test, test, test.
4. Make the switch. Flip on the new server, change IPs in the DNS.
5. Pull a final db copy from the old server to the new one for forums etc, shut down SQL on the old server to prevent orphans.
6. On the old server, enable a 307 redirect to the www2 URL.

This way, while DNS propagates around the net, your users are getting redirected to the new server. 307 is Google-safe, and it also instructs web browsers to re-post any form data to the new URL. In a few days, traffic to the old server will taper off to nothing and you can take it offline.

The only db-driven content downtime is the amount of time it takes to transfer the db and reattach it.

]]> By: Joshua Rushe http://theinnerlayer.softlayer.com/2007/ttl-and-propagation/#comment-58 Joshua Rushe Fri, 29 Jun 2007 23:35:42 +0000 http://theinnerlayer.softlayer.com/2007/ttl-and-propagation/#comment-58 Agreed, and that is a very good point. By far the best way to migrate a site is to leave both IPs live during propagation and monitor your logs. However, I've run into many, many instances where (typically due to finance issues) a company simply can't. If you are moving toward a strict cut over date in a situation such as that, this is the way to go. And for all those admins that aren't expiring records like they should--stop it. ;) Agreed, and that is a very good point. By far the best way to migrate a site is to leave both IPs live during propagation and monitor your logs. However, I’ve run into many, many instances where (typically due to finance issues) a company simply can’t. If you are moving toward a strict cut over date in a situation such as that, this is the way to go.

And for all those admins that aren’t expiring records like they should–stop it. ;)

]]> By: rackaid http://theinnerlayer.softlayer.com/2007/ttl-and-propagation/#comment-54 rackaid Fri, 29 Jun 2007 14:08:53 +0000 http://theinnerlayer.softlayer.com/2007/ttl-and-propagation/#comment-54 We do a lot of migration work. A common problem we see is that ISPs often have internal caching DNS servers that do not honor your TTL. We've found this to be the most problematic in Europe. Another key problem is search engines. We have seen it take as long as a week for a search engine to pick up your DNS change. Google used to take even longer but now they are pretty fast. Anytime you migrate a site, we advise leaving the old site in place, even if you must shutdown things like forums. You can then monitor the logs on the new and old location to assure all search bots are hitting your new site. We have seen pages get dropped from search engines from time to time because of internal DNS caching. We do a lot of migration work. A common problem we see is that ISPs often have internal caching DNS servers that do not honor your TTL. We’ve found this to be the most problematic in Europe.

Another key problem is search engines. We have seen it take as long as a week for a search engine to pick up your DNS change. Google used to take even longer but now they are pretty fast. Anytime you migrate a site, we advise leaving the old site in place, even if you must shutdown things like forums. You can then monitor the logs on the new and old location to assure all search bots are hitting your new site.

We have seen pages get dropped from search engines from time to time because of internal DNS caching.

]]> By: micksam7 http://theinnerlayer.softlayer.com/2007/ttl-and-propagation/#comment-52 micksam7 Thu, 28 Jun 2007 08:46:33 +0000 http://theinnerlayer.softlayer.com/2007/ttl-and-propagation/#comment-52 Ah yes, one of the fun things to do when moving servers. Being around small sites so long I've come familiar with "We're moving servers! If you see this page, please wait a few days to clear or go to newserver.site.wow". The few times I move servers I try to minimize the use of pages like that. With mirroring the site, the problem is always databases and forums. Either you can use purely static pages for a bit on the old server, leave the database as is and risk some posts and updates not getting transferred, or have the old server use the new server for database for a little while. [Or even have the old server act as a proxy to the new server, and many other options like database syncing/linking] Obviously the most desirable option would be having the old server connect to the new server's database. The problem with that becomes encryption. I'm sure you don't want all those database queries sent over the Internet in plain text. You have a few solutions to that however: Set your SQL server up to use SSL, use a SSL tunnel, or use secure VPN. For VPN I've used <a href="https://secure.logmein.com/products/hamachi/default.asp" rel="nofollow">Hamachi</a>, though I don't know how well Linux support is anymore. For SSL tunneling, you can use <a href="http://www.stunnel.org/" rel="nofollow">Stunnel</a>. Both are free [Hamachi has a "Premium" version however]. If you want to set your server to use SSL, good luck with that. Syncing files between servers can also be useful, or you can disable user uploads on the old server, if possible. I'm not sure what you can use for that though. If anyone has some ideas, please post and let us all know. One major issue with DNS which you didn't mention. Some ISP's DNS servers ignore the TTL and sometimes cache it up to a week. While rare, it can affect some customers. It is rare though, so it's not a -really big- issue. Note: If you have old server use new server's database, the queries may take a while to go through, so page generation time may increase a bit. Ah yes, one of the fun things to do when moving servers. Being around small sites so long I’ve come familiar with “We’re moving servers! If you see this page, please wait a few days to clear or go to newserver.site.wow”. The few times I move servers I try to minimize the use of pages like that.

With mirroring the site, the problem is always databases and forums. Either you can use purely static pages for a bit on the old server, leave the database as is and risk some posts and updates not getting transferred, or have the old server use the new server for database for a little while. [Or even have the old server act as a proxy to the new server, and many other options like database syncing/linking] Obviously the most desirable option would be having the old server connect to the new server’s database.

The problem with that becomes encryption. I’m sure you don’t want all those database queries sent over the Internet in plain text. You have a few solutions to that however: Set your SQL server up to use SSL, use a SSL tunnel, or use secure VPN. For VPN I’ve used Hamachi, though I don’t know how well Linux support is anymore. For SSL tunneling, you can use Stunnel. Both are free [Hamachi has a "Premium" version however]. If you want to set your server to use SSL, good luck with that.

Syncing files between servers can also be useful, or you can disable user uploads on the old server, if possible. I’m not sure what you can use for that though. If anyone has some ideas, please post and let us all know.

One major issue with DNS which you didn’t mention. Some ISP’s DNS servers ignore the TTL and sometimes cache it up to a week. While rare, it can affect some customers. It is rare though, so it’s not a -really big- issue.

Note: If you have old server use new server’s database, the queries may take a while to go through, so page generation time may increase a bit.

]]>