Database Queries and Scalability
I ran across this post with a very provocative title: (thanks Brent Ozar Unlimited). It’s an interesting read from a developer, talking about how so many website designs aren’t built for scalable because they rely too heavily on a set of database queries place on every page. If you expect a database query to retrieve or store a user id, or other data on every page, then you are limited to the scalability of your database back end. There’s no doubt there, but the author has ideas that can help you overcome these issues.
One of the problems is that systems read and write account information (or registrations) to the database directly. That’s definitely a scale issue, and the author suggests generating the client information on the web server and storing it in a more robust and scalable backend. That makes sense, but it’s with the caveat that ” As long as you eventually write the data to the db later on, you’re good to go.” Good point, but now you need a replication process of some sort (or messaging) that gets the data to your database. This ignores some of the synchronization issues across multiple servers as well, but it’s not a bad idea.
The idea of avoiding joins, by using a cached set of data to iterate through and pull data from other tables may work in some cases, but I’m not sure this is the most efficient way of grabbing data from a database, ensuring it’s the latest data, and not impacting concurrency. This technique seems to invalidate the power of an RDBMS by not using joins to effectively, and efficiently, retrieve data. I suspect replicated copies of slowly changing data is a better way of scaling your system than avoiding joins.
Overall I don’t love the article, and depending on which platforms and technologies you’re using, it might be better, or worse, advice. I would say that building better systems involves considering alternative techniques, allowing for failures, and using new techniques like messaging, caching, asynchronous processes and more. As we try to improve the quality of our software, we should be talking about, writing about, and trying out new techniques, learning what can work better in our environments.
Video and Audio versions
Today’s podcast features music by Everyday Jones. No relation, but I stumbled on to them and really like the music. Support this great duo at www.everydayjones.com.