About a month ago I kicked off a survey with some code to run to figure out how many log files your databases have (see here). There are all kinds of misconceptions about transaction logs and how to configure them (and how they work) and I'm amazed at the misinformation I continue to see published. For instance, a few weeks back I skimmed through a video that stated in multiple places that the default transaction log growth rate is 10MB – it's not, it's 10% and has been since SQL Server 2005.
I got data back from 1300 SQL Server instances across the world from 75 people (thanks!), with an average of 18 databases per instance including system databases (which jumped to an average of 29 per instance if I included 4 anomalous instances with many thousands of databases each).
Out of all those instances, only 32 had databases with more than one log file:
-
23 instances with one database with two log files
-
3 instances with two databases with two log files
-
1 instance each with 3, 4, 8, 9, and 27 databases with two log files
-
2 instances with one database with four log files
So 2.5% of instances surveyed had at least one database with more than one log file.
I think I'm pleased about that as I expected the number to be higher, but I also suspect I'm equating poorly configured VLF totals with general log mis-configuration, and a higher percentage of systems out there have transaction logs with too many VLFs. Let's settle for pleased :-)
But why do I care? And why should you care? Although it seems like there's nothing damaging about having multiple log files, I don't think that's true.
Firstly, having multiple log files implies that the first one ran out of space and because the second one still exists, the first one might still be pretty large (maybe the second one is large too!). I don't really care about that for crash recovery (which is bounded by how much unrecovered log there is), or performance of HA technologies like database mirroring, Availability Groups, or replication, which are bounded by transaction log generation rate, not size.
What I care about is performing a restore during disaster recovery. If the log files don't exist, they must be created and zero-initialized, and twice if you restore a diff backup too as both the full and diff restores zero out the log. If the first log file is as big as it can be, and there's a second log file still, that's potentially a lot of log file to zero initialize, which translates into more downtime during disaster recovery.
I would much rather that all the log files apart from the first one are removed once they're no longer needed, by simply waiting until all the active VLFs (marked with status 2 in the output from DBCC LOGINFO – see here) are in the first log file and then simply doing an ALTER DATABASE and removing the extra file, and then reducing the size of the remaining log file to something reasonable.
Which brings me to the second thing I care about: why was the extra log file needed in the first place? Why did the transaction log run out of space, necessitating creating another log file? That's the only explanation I can think of for having more than one log file as there is no performance gain from multiple log files – SQL Server will write to them sequentially, never in parallel (Jonathan demonstrates this neatly with Extended Events here.)
(I like to draw a parallel here with page splits. People fixate on the fact that they've got fragmentation, not the massive performance issue that created the fragmentation in the first place – page splits themselves!)
I blogged about the Importance of proper transaction log file size management more than three years ago (and here five years back), and many others have blogged about it too, but it's still one of the most common problems I see. Log growth can easily be monitored using the Log Growths performance counter in the Databases performance object and I'm sure someone's written code to watch for the log growth counter incrementing for databases and alerting the DBA.
For someone who's a DBA, there's no excuse for having out-of-control transaction logs IMHO, but for involuntary DBAs and those who administer systems where SQL Server sits hidden for the most part (e.g. Sharepoint), I can understand not knowing.
But now you do. Get reading these articles and get rid of those extra log files, and the need for them! Use the code in the original survey (see the link at the top) to see whether you've got an extra log files kicking around.
Enjoy!
PS For a general overview of logging, recovery, and the transaction log, see the TechNet Magazine article I wrote back in 2009.
12 Responses to Multiple log files and why they’re bad
Good, Proven and solid stand off against Multiple Log Files!
Thanks
hello,
I would like first to give you tons of thanks for your always interesting (and even more than that!) posts.
Regarding the size of Tlog, I’m afraid we (I mean DBA) don’t always have other choice than having big TLog, because of Index rebuild operations.
I manage some big databases that contains big tables with several billions of records that are not partionned (can’t rebuild at partition level). I let you imagine the size of Tlog (database is in full recovery mode as it concerns high critical business) when we have to rebuild Non clustered index (and in fact, I never rebuild clustered index in such tables…). That explain in my case why I get some hudge transaction log, and it doesn’t necessary means that it is an out-of-control transaction log.
Should I consider expand tlog before rebuild operation then shrink it after ?
O.
Good post, Paul. You mentioned the "Importance of proper transaction log file size management" – how easy people ignore this. And how easily it is to greatly reduce your RTO by ensuring your log is only as big as it needs to be.
And I have so many errors on that comment that I’m now ashamed of it. Jeez.
Thanks for sharing. Great info.
Hi Paul
i too have encountered the same issues with transaction log configuration on clients SQL server systems. In a previous contract the client had a 3rd party IT provider setup their Sharepoint system. The databases all had 1 file per core for the database files and for the transaction logs. I quizzed the architect who had implemented the system and he argued and pointed me to a Microsoft "best practice" document. I then pointed out to him that the doc stated that this was only for data files and not t-log and also that the doc was an optimisation for TempDB!!
In my current contract we have a large number of t-log files for each database, again, along the old 1 file per core configuration <yawn>
They even have 1 database that has 50 files in the primary filegroup all of differing sizes, growth and maxsize lol
Regards
Perry Whittle
in our case the SAN infrastructure dates back over 10 years and the drive volumes have been migrated to newer SAN’s with others added along the way. 10 years ago most of our SAN volumes were in the 10GB range. our databases were a lot smaller too.
so as the older volumes ran out of space we added new transaction log files to different drives. add SAN snapshots and replication to different volumes and its very hard to fix old mistakes
Hi Paul – Slightly off topic – But it is about the t-log ;) – why cant it be used for Instant File Initialization, but the data file can?
Apologies if you have blogged about this already.
Sorry for the delay replying to comments!
@arun Yes, I’ve blogged about it before – look in the Instant File Initialization category on my blog.
@olivier No – you should use index reorganize instead of rebuilding
Hi Paul,
Looking over the results of the survey would indicate that there are a large number of users that are still using a single log for TempDB or have intentionally excluded this information.
I know the common recommendation tempdb recommendation (as you pointed at in http://www.sqlskills.com/blogs/paul/post/A-SQL-Server-DBA-myth-a-day-(1230)-tempdb-should-always-have-one-data-file-per-processor-core.aspx) is about 1/2 – 1/4, even up to 1:1 on some machines with PAGELATCH issues.
Example; we run a server that has 16 cores and started it at 1/2:1 ratio giving us 8 tempdb log files. We have since adjusted based on watching our perf counters.
Our results are:
LogFiles Databases
1 9
8 1
NOTE: The last result is our TempDB
I’m curious what your take/guess is on why the results don’t show more single database with 2 or more log files?
Hi Paul,
great article as always!
we have a server which has about 300 DBs (of moderate size). we have a dedicated Log drive, which is configured to be of 100GB. In most of those databases we have a single log file. However there are a few databases, which have lots of activity and some big tables. to accomodate for the small size of the drive, i have created secondary log files on other drives. this is purely done to be cautios, if the log drive fills up, the application does not stop. this only happens once in a year, and therefore does not warrant a bigger log drive.
James – multiple files for tempdb only applies to data files, not log files.