Setting mentoring ground rules

One of the things I always recommend in the first newsletter of each year is to try to get a mentor. On the flip side of that is that one of the most rewarding things you can do in your professional life is to *be* a mentor to one or more people. I mentored more than 50 people back in 2015, and I think I’m going to do some more mentoring, starting in the Spring (watch the blog for details – but definitely not 50 again!).

If you’re going to be a mentor, I think it’s very important to set ground rules. One of my mentees from 2015 just sent me email saying he’s going to be mentoring this year (*so* cool to see people paying it forward!) and asking about the ground rules I set. I’ve been mentoring people for almost 20 years and I think you need to cover at least:

  • How communication will work, and expectations on both sides for how long a response may take
  • How to end the relationship
  • Which areas are fair game, and which aren’t (e.g. I wouldn’t answer SQL Server questions with the 2015 group)
  • Statement of confidentiality and trust
  • What actions will cause the relationship to end
  • How not to react to things I’ll say (e.g. I’m very blunt and honest – the whole point of a mentoring relationship is clear feedback)

That way there’s no ambiguity and you both know how things are going to work.

As an example, here’s the set of rules I sent to all my mentees to ‘ok’ before I’d move forward with them:

Lots of housekeeping, ground rules, and expectation setting in this email so PLEASE read through this email and let me know whether you’re comfortable with everything I say below. If not, for any reason, just say so (you don’t need to say why) and we’ll walk away with no hard feelings. I also need to know that you’re getting my emails. Once I hear back from you, I’ll send the kick-off part 2 email.

As I mentioned in the original blog post, I do all mentoring over email. I have two reasons for that:

It makes the whole process asynchronous, which means there’s no immediate pressure to respond and so is easier on us all. It also means we can think about responses for a week or two, and that’s especially useful as I’ve got several long trips without Internet this year.

I really don’t like talking on the phone. Phones are a necessary evil, but they don’t make for well-though-out discussions.

Once I’ve had a response from you so I know my email’s getting through, whenever I send an email out, I won’t chase you up to reply to it. If you go dark for, say, a month, I’ll assume you’re no longer interested or you’re done with mentoring. I’d appreciate a note to that effect if you decide that though, so we get a positive closure.

You can expect responses from me within a week or two. If a longer response will be delayed, I’ll let you know. If it goes more than two weeks and you haven’t heard from me about a reply, please ping me as your email might have been lost or (rarely) I might have accidentally deleted or mis-filed it.

I take all mentoring relationships seriously, and I expect you to as well. You’re going to be opening up and telling me things about yourself that you might not share/have shared with anyone else (such as hopes, failures, negatives, positives, weaknesses, strengths, work problems, family problems, whatever). Anything you tell me will remain strictly between us, and only us.

You can publicize whatever you want about the mentoring relationship and any advice I give you, unless I (rarely) tell you something is in confidence.

Do not tell me anything that is illegal or unethical (e.g. you cheated on a certification) – that will cause me to end the mentoring relationship immediately. No second chances on this. If there’s something we’re discussing that is making me uncomfortable, I’ll let you know.

I will not discuss politics or religion 1-1 through email. Sheep, diving, and electronics are fair game though :-)

We’re going to get to know each other better over the rest of this year. Feel free to Facebook friend me if you want – no pressure – I don’t care either way, but some people don’t like to presume to ask.

I will be blunt and honest with you; I don’t sugar-coat things. You’ve probably already seen that from me on Twitter and on my blog. Getting straight to the point in any discussion is the easiest way to get a point across clearly and with little or no ambiguity. Please don’t ever take anything I say to you as trying to be nasty, belittling, or destructive criticism – that’s not how I work or who I am. The whole point of a mentoring relationship is to be constructive, so please take everything I say with that in mind. If you find yourself having a hard time with that, let me know and we can work on that too. And feel free to disagree with anything I say too, you won’t offend me.

As I mentioned in the original post, this isn’t about me answering SQL Server questions for you. That’s what forums, Twitter, etc are for. Although I can help you figure out how to grow skills, this isn’t a mostly technical conversation.

This may seem like a lot of rules, but I’m very easy going. At the same time I like to have expectations set and ground rules agreed to so we all know where we stand – this is a big time investment from all of us, so it pays to get everything out in the open right at the start.

If that all sounds cool to you, let me know and I’ll send the second part of the kick off.

Thanks!

Feel free to use any or all of this if you’re going to be a mentor, or ask for something like this if you’re going to be a mentee.

The main thing to bear in mind is that both mentoring and being a mentee should be rewarding, and not feel like a chore or a slog. Setting some rules up front will help define the relationship and provide a clear way to end it if things aren’t going well.

Enjoy!

More live online training from Kimberly in March!

Due to the popularity of Kimberly’s delivery of her new IEVLT: Immersion Event on Very Large Tables: Optimizing Performance and Availability through Partitioning in January, and the fact that many of you told us that you didn’t have a chance to attend, we’re running it again in March! The class will be delivered live via WebEx on March 20-22 (roughly 12-13 hours of content plus open Q&As; about the same as two full workshop days!) and the recording is available to attendees for six months following the end of the class. Note: we’re not planning any more deliveries of IEVLT until 2019.

Here are some quotes from people who attended the January class:

  • “Kimberly is incredibly knowledgeable and was able to adapt the techniques to all the different scenarios presented to her.”
  • “The best educator I’ve ever seen. She makes complex concepts “magically” easy to grasp. Incred-amazing.”
  • “Great course. I could hear clearly, the content was relevant to current day problems, and provided clear instruction.”
  • “This was REALLY good. Getting to an IE is tough there are only a few a year and more importantly because there are only a few they cover a pretty broad range of information. Since I do mainly database design and query tuning, I can’t justify much beyond IE1 as I don’t do day to day DBA work. Seeing you were offering the online event focused on a specific topic – very large tables – was PERFECT. I know I really need to improve my knowledge of the options in that area. I recalled the PV/PT architecture from IE1 and knew a refresher, coupled with the new information would be perfect. The cost was BEYOND reasonable. The time frame, at only about 1/2 a day, was easy to justify and easy to manage keeping up with regular work while I did it. So this worked out to be a perfect event.” – Todd Everett, USA
  • “SLIGHTLY prefer the classroom option, but attendance to a location can be very hard to manage and therefore WebEx is a VERY, VERY good alternative. I didn’t think it would be, but I’m definitely sold.” – Kevin Urquhart, UK
  • “I’m really glad I decided to participate in this event. I learnt a lot and I already see lots of opportunities to implement some of the strategies at my workplace (after working through all the materials of course :P).” – Ernst Kruger, South Africa
  • “Loved the online aspect. It felt like I was there with the question ability and having the questions just answered right there. I felt I had a voice and could ask anything and the ability to watch it later made it totally worth the registration.”
  • “Always love to sit at the feet of Kimberly. Always learn a ton.”
  • “I struggled with WebEx at first… felt a little like watching a video… BUT the moment it became interactive (which was very early on), that whole perception changed. By the end I REALLY enjoyed it as a method.”
  • “I really enjoyed the ability to ask questions as the course went along so that I didn’t forget what I wanted to ask while you were teaching. This allowed for questions to come through and class to continue until a good stopping point to answer the questions. Plus having the questions written from other attendees was nice for future reference instead of trying to remember from an in-person class discussion.”
  • “Kimberly, I have to tell you that you are hands down the finest educator I have ever met. Your dynamic personality, combined with your depth of knowledge, and passion to fully understand the subject matter are unmatched, in my experience. I feel truly privileged to have had the opportunity to learn from you again and again. You really DO make learning fun. Thank you!” – Mike Petri, USA
  • “I learned a lot of new topics and found this class really useful. Usually when I attend a class or an in person event I learn new things but no where near this class. Thanks for a great class.”

Rather than have people try to watch a full day of training at their computer for one of more days, the class will run from 10am to 3pm PST each day, with two 90-minute teaching sessions, each followed by Q&A, and a lunch break. We chose to do this, and to spread the class over a few days, so the times work well for those in the Americas, Africa, and Europe. We also realize that this is complex, architectural content, so want to give attendees time to digest each day’s material, plus extensive Q&A.

The modules covered will be:

  • Horizontal Partitioning Strategies
  • Partitioned Views
  • Partitioned Tables
  • Tables, Indexes, Keys, and Relationships
  • Implementing the Sliding Window Scenario
  • Key Partitioning Concerns and Considerations
  • Partitioning Techniques Combined
  • Review

The price of the class is US$895 and you can get all the details here.

The class was also announced in our newsletter this morning, with a $100 discount for those people who received that newsletter, valid for two weeks. All future live, online classes will always feature a two-week discount for newsletter subscribers.

We decided to start teaching some live, online classes as we recognize that not everyone can travel to our in-person classes, or take that time away from work or family, or simply have travel budget as well as training budget. People also have different ways they learn, some preferring in-person training, some preferring recorded, online training, and some preferring live, online training.

We’ll be doing more of these in 2018, on a variety of topics, so stay tuned for updates (and discounts through the newsletter).

We hope you can join us!

SQLskills SQL101: Why does my heap have a bunch of empty pages?

SQLskills has an ongoing initiative to blog about basic topics, which we’re calling SQL101. We’re all blogging about things that we often see done incorrectly, technologies used the wrong way, or where there are many misunderstandings that lead to serious problems. If you want to find all of our SQLskills SQL101 blog posts, check out SQLskills.com/help/SQL101.

Here’s a question I’m often asked (paraphrasing):

I’ve got a large heap table where the space isn’t being given up when I delete a large number of records, but then when I shrink the database the heap is reduced in size. Can you explain?

This behavior is how SQL Server works, but it’s pretty non-intuitive. When a page in an index becomes empty, it’s always deallocated, as an empty page isn’t allowed in a SQL Server index structure from SQL Server 2005 onward. However, the structure of a heap is different and as a result, the behavior is too.

Whenever a row is deleted in a heap, it’s usually the case that the page containing the row does not become empty. However, if the heap page that the row is stored on becomes empty as a result of the delete, the page cannot be deallocated from the table unless an exclusive (X) table lock is held (to remove the page from the internal free-space “tracking” mechanism that is in place for heaps). It’s not common that a table level X lock is held unless lock escalation has occurred (because you’re deleting enough rows to trigger escalation from individual row X locks to a single table X lock), or if you specifically use the TABLOCK hint on the delete statement, for instance. But, because both of these circumstances are unlikely, the empty heap page usually cannot be deallocated.

There is a Knowledge Base article that describes this phenomenon: KB 913399. However, the KB article only references up to and including SQL Server 2005 but this behavior exists in all version and is very easy to reproduce if you want to prove it to yourself.

The script below will create a database, create a table with one row per heap page, show you how many pages and rows, and how full each page is. Next it’ll delete all the rows, and show you the pages are still there, with no rows, and empty. Give it a try!

CREATE DATABASE [EmptyHeapTest];
GO
USE [EmptyHeapTest];
GO

CREATE TABLE [test] ([c1] INT IDENTITY, [c2] CHAR (8000) DEFAULT 'a');
GO

SET NOCOUNT ON;
GO
-- Insert 1000 rows, which will allocate 1000 pages
INSERT INTO [test] DEFAULT VALUES;
GO 1000

-- How many pages and how full?
SELECT [page_count], [record_count], [avg_page_space_used_in_percent]
FROM sys.dm_db_index_physical_stats (DB_ID (N'EmptyHeapTest'), OBJECT_ID (N'test'), 0, 	DEFAULT, 'DETAILED');
GO
page_count           record_count         avg_page_space_used_in_percent
-------------------- -------------------- ------------------------------
1000                 1000                 98.974549048678
-- Delete all the rows
DELETE FROM [test];
GO

-- Did all the pages get deleted?
SELECT [page_count], [record_count], [avg_page_space_used_in_percent]
FROM sys.dm_db_index_physical_stats (DB_ID (N'EmptyHeapTest'), OBJECT_ID (N'test'), 0, 	DEFAULT, 'DETAILED');
GO 
page_count           record_count         avg_page_space_used_in_percent
-------------------- -------------------- ------------------------------
1000                 0                    0

The empty pages will be reused by subsequent inserts, but if the space isn’t going to be reused following a large delete in a heap, you might consider using the TABLOCK hint to allow the empty pages to be deallocated and the space made available for other objects in the database to use.

Another alternative is to just use a clustered index instead, or if a heap is necessary, you could rebuild the heap using ALTER TABLE … REBUILD (that was added in SQL Server 2008 to support enabling compression on a heap), with the caveat that this will cause all the table’s nonclustered indexes to be rebuilt.

On the extreme end (in my opinion), you could reclaim the empty heap space using a shrink operation. Shrink won’t free up space inside pages as it moves them (with the exception of compacting LOB pages as it goes – somewhat unsuccessfully depending on which version and build you’re on – see KB 2967240), but it will remove empty pages rather than moving them. This will effectively shrink the heap after a large delete, but with the usual caveats about shrink causing index fragmentation and generally being an expensive, slow operation to perform.

So now you know why you may have a bunch of empty pages in your heaps!

Calling all user group leaders! We want to present for you in 2018!

By the end of December, we at SQLskills had presented remotely (and a few in-person) to more than 100 user groups and PASS virtual chapters around the world in 2017, and we already have 45 user group sessions scheduled in 2018!

We’d love to present remotely for your user group in 2018, anywhere in the world. It’s not feasible for us to travel to user groups or SQL Saturdays unless we’re already in that particular city, but remote presentations are easy to do and are becoming more and more popular. We haven’t had any bandwidth problems doing remote presentations in 2017 to groups as far away as South Africa, Australia, and New Zealand, plus Norway, Canada, UK, Poland, Belgium, Brazil, Czech Republic, and Ireland. This way we can spread the community love around user groups everywhere that we wouldn’t usually get to in person.

Note: we have our own Webex accounts which we generally use, or we can use your GoToMeeting or Webex, but you must use computer audio – we won’t call in by phone as the sound quality is too poor. We also will not use Skype/Lync as we’ve had too many problems with it around user group laptops and sound.

So, calling all user group leaders! If you’d like one of us (me, Kimberly, Jon, Erin, Glenn, Tim) to present remotely for you in 2018 (or maybe even multiple times), send me an email and be sure to include:

  • Details of which user group you represent (and if sending from a shared user group account, your name)
  • The usual day of the month, meeting time, and timezone of the user group
  • Which months you have available, starting in January 2018 (a list of available dates would be ideal)
  • Whether you’d like just one or multiple

And I’ll let you know who’s available with what topics so you can pick. We have around 20 topics across the team that we can present on.

What’s the catch? There is no catch. We’re just continuing our community involvement next year and we all love presenting :-)

There’s no deadline for this – send me an email at any time and we’ll see what we can do.

We’re really looking forward to engaging with you all!

Cheers

PS By all means pass the word on to any SharePoint and .Net user group leaders you know too.

SQLintersection Spring 2018

As we head towards our 11th SQLintersection in March, I’m excited to say that it’s once again our most diverse, complete, and information-packed show yet!

One of the pieces of feedback we hear over and over is that attendees love SQLintersection because it’s a smaller, laid-back show, where you get to actually spend time talking with the presenters 1-1. I have to say that’s one of the reasons why we love the show so much; *we* get to spend time talking to attendees, rather than being mobbed by hundreds of people after a session ends. And we only pick presenters who we know personally, and who we know to be humble, approachable, and eager to help someone out.

We have 2 pre-con days at the show and with our post-con day, there are 8 full-day workshops from which to choose. We have 40 technology-focused (NOT marketing) sessions from which to choose, plus two SQL Server keynotes, multiple industry-wide keynotes by Microsoft executives, and the ever-lively closing Q&A that we record as a RunAs Radio podcast (you can listen to the recording from our Fall 2017 show here.)

You’ll learn proven problem-solving techniques and technologies you can implement immediately. Our focus is around performance monitoring, troubleshooting, designing for scale and performance, cloud, as well as new features in SQL Server 2014, 2016, and 2017. It’s time to determine your 2008 / 2008 R2 migration strategy – should you upgrade to 2016/2017 directly? This is the place to figure that out!

If you’re interested in how we got here – check out some of Kimberly’s past posts:

  1. SQLintersection: a new year, a new conference
  2. SQLintersection’s Fall Conference – It’s all about ROI!
  3. Fall SQLintersection is coming up soon and we can’t wait!
  4. SQLintersection Conference and SQLafterDark Evening Event – what a fantastic week in Vegas

And Kimberly recorded a Microsoft Channel 9 video where she discusses the 2017 Spring show – see here.

SQLafterDark

With minimal to no marketing filler, we’ve largely kept our conference focus on ROI and technical content (performance / troubleshooting / tales-from-the-trenches with best practices on how to fix them ) but we’ve also added even more social events so that you really get time to intersect with the conference attendees and speakers. The addition of the SQL-specific, pub-quiz-style evening event SQLafterDark was wildly popular from some of our past shows and that’s returning for Spring!

 

SQLintersection: Great Speakers!

Once again, I think a great show starts with great speakers and current / useful content. All of these speakers are industry-experts that have worked in data / SQL for years (some can even boast decades) but all are still focused on consulting and working in the trenches. And, they’re good presenters! Not only will you hear useful content but you’ll do so in a way that’s digestible and applicable. Every speaker is either an MCM (Master), a past/present SQL Server MVP, or a past/present Microsoft employee (or a combination of all three!) But, regardless of their official credentials – ALL are focused on providing the most ROI that’s possible in their session(s) and/or their workshops, and ALL have spoken for SQLintersection multiple times.

Check out this phenomenal list of speakers:

  • Aaron Bertrand – MVP, SentryOne
  • Ben Miller, MCM, MVP, past Microsoft, Consultant
  • Bob Ward, SQL Server team at Microsoft
  • Brent Ozar, MCM, past-MVP, Consultant
  • David Pless – MCM, Microsoft
  • Gareth Swanepoel, Microsoft
  • Glenn Berry, past-MVP, SQLskills
  • Jes Borland, past-MVP, Microsoft
  • Justin Randall, MVP, SentryOne
  • Tim Chapman – MCM, Microsoft
  • Tim Radney – MVP, SQLskills
  • Kimberly L. Tripp – MCM Instructor, MVP, past SQL Server team at Microsoft, SQLskills
  • Paul S. Randal – MCM Instructor, MVP, past SQL Server team at Microsoft, SQLskills
  • Pedro Lopes, MCM, SQL Server team at Microsoft

You can read everyone’s full bio on our speaker page here.

SQLintersection: When is it all happening?

The conference officially runs from Monday, March 26 through Wednesday, March 28 with pre-conference and post-conference workshops that extend the show over a total of up to 6 full days. For the full conference, you’ll want to be there from Saturday, March 24 through Thursday, March 29.

  • Saturday, March 24 – pre-con day. There are three workshops running:
    • Migrating to SQL Server 2017 with Glenn Berry
    • SQL Server Indexing Strategies for Performance with Kimberly L. Tripp
    • The T-SQL Programming Workshop with Tim Chapman
  • Sunday, March 25 – pre-con day. There are three workshops running:
    • Azure for the SQL Server DBA with Tim Radney
    • Performance Troubleshooting using Waits and Latches with Paul S. Randal
    • PowerShell Essentials for the DBA with Ben Miller
  • Monday, March 26 through Wednesday, March 28 is the main conference. Conference sessions will run all day in multiple tracks:
    • Check out our sessions online here
    • Be sure to check out our cross-conference events and sessions
    • Get your pop-culture trivia and techie-SQL-trivia hat on and join us for SQLafterDark on Tuesday evening, March 27
  • Thursday, March 29 is our final day with two post-conference workshops running:
    • Mastering Query Tuning for SQL Server with Brent Ozar
    • SQL Server 2016 / 2017 and Power BI Reporting Solutions with David Pless

SQLintersection: Why is it for you?

If you want practical information delivered by speakers that not-only know the technologies but are competent and consistently, highly-rated presenters – this is the show for you. You will understand the RIGHT features to troubleshoot and solve your performance and availability problems now!

Check us out: www.SQLintersection.com.

We hope to see you there!

PS – Use the discount code ‘SQLskills’ when you register and receive $50 off registration!

New Pluralsight course: Automatic Tuning in SQL Server 2017 and Azure SQL Database

Erin’s latest Pluralsight course has been published – SQL Server: Automatic Tuning in SQL Server 2017 and Azure SQL Database. It’s 90 minutes long, and from the course description:

There’s a whole new world of automated features in SQL Server, based on data, that are designed to make life easier for the data professional, and it’s time to start using them! Microsoft has invested a lot of effort in these tools that can automate areas of performance tuning and give you time to work on more interesting and creative tasks that can’t be automated. In this course, SQL Server: Automatic Tuning in SQL Server 2017 and Azure SQL Database, you’ll learn how Query Store has been upgraded to also collect wait statistics from queries, to allow you to identify performance bottlenecks at the query level and dig deeper to resolve them. Next, you’ll explore Automatic Plan Forcing, which allows SQL Server to identify queries where performance has regressed and automatically force a prior query plan that is more efficient. Finally, you’ll take a dive into Automatic Index Management, which gives SQL Server the capability to automatically create missing indexes and remove unused or duplicate indexes. By the end of this course, you’ll have the skills and knowledge to start enabling these features in appropriate parts of your SQL Server environment and taking back some of your precious time!

The modules are:

  • Introduction
  • Finding and Using Wait Statistics in Query Store
  • Using Automatic Plan Correction
  • Using Automatic Index Management

Check it out here.

We now have more than 170 hours of SQLskills online training available (see all our 56 courses here), all for as little as $29/month through Pluralsight (including more than 6,000 other developer and IT training courses). That’s unbeatable value that you can’t afford to ignore.

Enjoy!

2017 review: the year by the numbers

The last post of the year! It’s been a really excellent year all round and time for my traditional post counting down some of the numbers that have been my life this year.

  • 106,914: the number of miles I flew on United this year (plus 8,504 miles on other airlines around Indonesia)
  • 34,827: my current tweet total (up 742 from 2016)
  • 13,642: the number of subscribers to our Insider mailing list (up 208 from 2016)
  • 13,379: the number of people who follow my Twitter ramblings (up 1,211 from 2016)
  • 9,683: the number of emails I sent (up 885 from 2016 – not a good thing!)
  • 1,900: the number of non-reference books (real ones) that I own (up 121 from 2016)
  • 1,059: the number of books I own but haven’t read yet (up 61 from 2016, but I removed a bunch of reference books from the total)
  • 329: the number of SQL Server wait types I have documented in my Wait Types Library (up 43 from 2016)
  • 141: the number of nights away from home (down 43 from 2016, and all with Kimberly)
  • 172: the total number of hours of online training we have available on Pluralsight (up 23 from 2016)
  • 153: the number of dives I did this year, in Indonesia and the Bahamas, taking my total to 805
  • 107: the number of minutes of my longest dive this year, at what we think is the best muck-diving site in the world
  • 38.63: the percentage of time we were away from home (which is why we call it our vacation home!)
  • 60: the number of SQLskills blog posts, including this one
  • 58: the number of Pluralsight courses we have available
  • 48: the number of books I read (see this post)
  • 44: the number of flights this year (2 more than 2016)
  • 42: the answer to the question of life, the universe, and everything!
  • 35: the number of days in Immersion Events and conferences
  • 24: the number of different places we slept apart from our house and on planes
  • 18: the number of monthly magazines I subscribe to
  • 17: the number of airports I flew through this year
  • 10: the number of years I’ve been married to Kimberly
  • 9: the number of different octopus species we saw in Indonesia this year (coconut, algae, reef, blue-ring, greater blue-ring, starry-night, mimic, wonderpus, long-arm)
  • 7: the number of full-time SQLskills employees, all of whom are fabulous and indispensable
  • 5: the number of Great Courses I watched this year
  • 4: the number of countries we visited this year
  • 3: the number of new airports I flew through (Ambon, Indonesia; Freeport, Bahamas; Fort Lauderdale), taking my total to 94
  • 2: the number of awesome daughters we have
  • 1: the number of new bird species I saw, taking my total to 512
  • 1: the number of new airlines I flew on (WestJet), taking my total to 36
  • 1: the person who seems to cram the most into non-work time (farming, scouts, PokemonGo, building, …): Tim Radney
  • 1: the person who is the best at snapping her fingers and ran a marathon this year: Erin Stellato
  • 1: the biggest hardware geek, master beer brewer, and ex-tank commander I know: Glenn Berry
  • 1: the number of Jonathan Kehayias in the world – thankfully :-)
  • 1: the number of indispensable assistants, without whom our lives would be a distressing quagmire – Libby we love you!
  • 1: our wonderful chocolate lab, Coco, who passed away in November
  • Finally, the one and only best person in my life: Kimberly, without whom I would be lost…

Thank you to everyone who reads our blogs, follows us on Twitter, sends us questions, watches our videos, comes to our classes, and generally makes being deeply involved in the SQL community a joy.

I sincerely wish you all a happy, healthy, and prosperous New Year!

Cheers!

(July 4th, on our deck, watching the fireworks over Lake Sammamish)

(July 8th, in Canada: It’s not dark. It’s 150 miles to home. We have a giant letter R. We’re both wearing sunglasses… let’s go!)

(Later in July, diving in the Bahamas with Emma, the 18-foot tiger shark; I’m on the left, our daughters are on my right)

2017: the year in books

Back in 2009 I started posting a summary at the end of the year of what I read during the year (see my posts from 2009201020112012201320142015, 2016) and people have been enjoying it, so here I present the 2017 end-of-year post. I set a moderate goal of 50 books this year but I only managed 48 as I spent a bunch of time in the first half of the year catching up with my magazine backlog. Next year I’m setting myself a goal of reading 50 books again.

For the record, I read ‘real’ books – i.e. not in electronic form – I don’t like reading off a screen. Yes, I’ve seen electronic readers – we both have iPads – but I don’t like reading electronically. Having said that, I did read one electronically this year out of necessity, and will have to read a few in 2018 electronically due to insurmountable luggage weight and volume restrictions on one of our trips. I also don’t ‘speed read’ – I read quickly and make lots of time for reading.

Why do I track metrics? Because I like doing it, and being able to compare against previous years. Some people don’t understand the logic in that – each to their own :-)

I went back-and-forth over the last few days about which book to nominate as my favorite, and I just couldn’t come to a decision, so just like in most years, I give you my favorite 3 books: The North Water by Ian McGuire, I Contain Multitudes: The Microbes Within Us and a Grander View of Life by Ed Yong, and Remarkable Books: The World’s Most Beautiful and Historic Works by DK. All three are superb books and I strongly recommend you give them a try. You can read my review of them in the top-10 list below.

Now the details. I enjoy putting this together as it will also serve as a record for me many years from now. I hope you get inspired to try some of these books – push yourself with new authors and very often you’ll be surprisingly pleased. Don’t forget to check out the previous year’s blog posts for more inspiration too.

As usual I leave you with a quote that describes a big part of my psychological make-up:

In omnibus requiem quaesivi, et nusquam inveni nisi in angulo cum libro!

Analysis of What I Read

I read 20,628 pages, which is an average of 56.52 pages a day. The chart below shows the number of pages (y-axis) in each book I read (x-axis).

The average book length was 430 pages, almost exactly the same as last year (429) but shorter than previous years. That’s because I again read a lot of series books where each isn’t hugely long.

I read a lot more nonfiction than in previous years, which is shown by 8 of my top-13 below being nonfiction books.

The Top 10

I read a lot of excellent books this year and I thought as I only read 48 that I’d be able to whittle them down to a top-10, but I couldn’t, so here is my top 13. If you don’t read much, at least consider looking at some of these in 2017. It’s impossible to put them into a priority order so I’ve listed them in the order I read them, along with the short Facebook review I wrote at the time.

 #3; The Bone Tree; Greg Iles; 816pp; Fiction; January 10; (This is the follow-on from the fantastic Natchez Burning, which was my favorite book of 2014, and you really need to read that before this one, as this one continues the story from the next day. This book starts to pull the characters into the Kennedy assassination investigations and links to the extreme racist murders from the 1960s. It’s hard to say more without giving away some of the plot and twists. The writing is excellent, with great dialog, and the book is a real page-turner. The story isn’t complete though – that’s coming in March in Iles’ conclusion to the trilogy: Mississippi Blood. Highly recommended!)

 #10; Dirty Wars: The World Is a Battlefield; Jeremy Scahill; 681pp; Nonfiction; April 21; (This is an extremely interesting book that investigates the ongoing, autonomous U.S. military operations around the world that have been undertaken as part of the war on terror. It exposes the extrajudicial capture/torture/kill program that was created in secret by the Bush administration (and continued and greatly enhanced by the Obama administration), and operated by the CIA and various Special Forces under the JSOC umbrella in Yemen, Iraq, Pakistan, Afghanistan, and Somalia. It uses the case of Anwar Awlaki as the center-point of the narrative. He was a U.S. citizen of Yemeni descent, who became an Islamic teacher and, after fleeing the U.S., went on to become an outspoken proponent of violent jihad against the West. He was linked to AQAP and wrote for their Inspire newsletter. I actually downloaded one of the newsletters to read – quite disturbing stuff. It’s a huge book that’s full of factual information without engaging in partisan criticism. Scahill raises some excellent questions about the morality and constitutional legality of such a program, especially when it targets U.S. citizens for death. Whether you agree with such programs or not, the book is well worth reading. Highly recommended!)

 #14; The Revenge of Analog: Real Things and Why They Matter; David Sax; 304pp; Nonfiction; July 11; (What a great book! As a confirmed Luddite, I was really looking forward to the various parts of the book. It covers the resurgence of vinyl records, paper and magazines (check out Stack and Delayed Gratification, both of which I subscribed too), photo film, board games, books, bricks-and-mortar retail, and analog ideas around work, education, and online companies. It’s quite clear that going all digital isn’t what people want. Very interesting, not written in a preachy or manifesto way, and highly recommended!)

 #18; Fast Food Nation: The Dark Side of the All-American Meal; Eric Schlosser; 362pp; Nonfiction; August 25; (Schlosser’s book is a searing expose of the business practices behind the American fast food industry, focusing on McDonald’s and the giant beef processing companies that supply it. Although written in 2001, the industry is still much the same today (not including those chains that create and cook their food fresh every day). The most shocking thing is really how the industry has pressured the government into reducing and/or removing safety and health inspections from the beef supply chain. Once you read about the meat processing facilities, I’ll be surprised if you ever eat a frozen beef patty again. This is definitely not a go-vegetarian rant, but a level-headed, fact-based examination of how fast food is sourced, processed, and prepared. Highly recommended. (And I also strongly recommend Schlosser’s Command and Control that I read a few years ago.))

 #22; What the Dog Saw: And Other Adventures; Malcolm Gladwell; 448pp; Nonfiction; September 3; (I really enjoy Gladwell’s books (e.g. The Tipping Point, Blink, Outliers) and this one was just as good. Rather than being a book that focuses on a single topic about how to understand ourselves and the world, this is a collection of some of Gladwell’s best long articles for The New Yorker along the same lines. It covers 19 topics, all of which are extremely interesting and thought provoking. Highly recommended! (and all his other books too!))

 #23; Crystal Fire: The Invention of the Transistor and the Birth of the Information Age; Riordan and Hoddeson; 368pp; History; September 3; (This book recounts the history of the development of the transistor, from the beginnings of solid-state physics in the late 1800s and early 1900s through to the invention and fabrication of the first microchips in the early 1960s. It’s also something of a biography of William Shockley, who was arguably the central character in the transistor’s creation, along with John Bardeen and Walter Brattain, fellow Bell Labs employees and co-winners of the 1956 Nobel Prize for Physics. Hugely interesting and very readable – highly recommended!)

 #27; Why Nations Fail: The Origins of Power, Prosperity, and Poverty; Acemoglu and Robinson; 544pp; Nonfiction; September 9; (The book puts forth a theory that nations with inclusive economic and political institutions are much more likely to succeed than those with extractive institutions (extractive meaning that the populace is exploited by a small elite for their own gain, inclusive meaning that all citizens are treated equally and property rights are protected by law). It also explains why inclusive institutions developed in some parts of the world and not in others, with exploitative colonialism being a major historical impediment to inclusivity. Lots of interesting case studies and history – highly recommended!)

 #30; The North Water; Ian McGuire; 272pp; Historical Fiction; September 26; (Excellent book! (So good I just picked up all McGuire’s other novels.) Set in the mid-1800s as the whaling industry is coming to an end. A ship sets off for the Greenland waters and many dark things happens. It reminds me of Melville’s Moby Dick, but a lot faster, and a lot more raw. Highly recommend and a possible candidate for my best book of the year!)

 #31; I Contain Multitudes: The Microbes Within Us and a Grander View of Life; Ed Yong; 357pp; Nonfiction; October 1; (What an excellent book! Thanks to Buck Woody for the recommendation. The book explains all kinds of fascinating things about bacteria and our microbiomes, the history of their understanding by science, and how the quest to rid ourselves of them is futile, and in quite a few cases, actually harming people that live in developed nations. More interestingly, it also introduces many scientists who have identified beneficial bacteria and used them to help solve medical or environmental problems – e.g. preventing the dreaded Bd fungus killing a group of frogs, inoculating a group of mosquitoes with a bacteria that prevents them carrying dengue fever, and fecal-matter transplants to help sufferers of persistent C-diff infections – poo pill anyone? Hugely interesting, and written in an entertaining manner, this book is for anyone. Highly recommended!)

 #36; Days Without End; Sebastian Barry; 272pp; Historical Fiction; October 13; (This is my time reading Barry’s work and won’t be the last (already bought 3 more of his books). The book follows a pair of Irish immigrants who join the US army in the mid-1800s, take part in brutally putting down Indians, and fight in the Civil War on the Union side. Very atmospheric and a page turner. Highly recommended!)

 #37; The Lighthouse Road; Peter Geye; 304pp; Historical Fiction; October 14; (Excellent novel, following a misbegotten family in early 1900s north Minnesota – starting with a young Norwegian immigrant woman. Logging, wolves, fishing, orphans, tough times, and heartbreak, lots of heartbreak. Highly recommended.)

 #44; The Little Paris Bookshop; Nina George; 416pp; Fiction; November 18; (I started this last week on the day Coco died as a distraction – what could be more uplifting for a bibliophile than a book about books and a bookshop? It’s a lovely book, about a bookseller who prescribes certain books to ease his customers’ pains. He loses love, takes his floating bookshop downriver, and finds it again. Gentle read and very relaxing. And did I mention it’s about books? Recommended.)

 #47; Remarkable Books: The World’s Most Beautiful and Historic Works; DK; 256pp; Nonfiction; December 25; (Merry Christmas! Just finished book #47 this year. As you all know I’m seriously addicted to books and reading – a true bibliophile. I’ve been working through this wonderful book during December reading about a few books each day. Here’s the Amazon blurb about it that describes is better than I can:<begin> A beautifully illustrated guide to more than 75 of the world’s most celebrated rare and seminal books and handwritten manuscripts ever produced with discussions of their purpose features and creators. From ancient masterpieces such as The Art of War written on the leaves of bamboo to the stunningly illustrated Birds of America to Chairman Mao’s Little Red Book Remarkable Books delves into the stories behind the most incredible tomes ever produced offering an insight into their wider social and cultural context and is chronologically ordered to demonstrate the synergies between the growth in human knowledge and the bookmaking process. Alongside breathtaking images of the books and manuscripts themselves close-up views draw out interesting features which are discussed in greater detail while biographies tell the lives of the people who produced them. This coffee table–worthy book is wrapped in a textured jacket with gold foil making it a great gift for those with an interest in literature and art and design. <end> I’m familiar with many of the books covered especially the illustrated manuscripts from the Middle Ages as those are of particular interest to me. It really is a fantastic book itself and I hugely recommend it!)

The Complete List

And the complete list, with links to Amazon so you can explore further. One thing to bear in mind, the dates I finished reading the book don’t mean that I started, for instance, book #2 after finishing book #1. I usually have anywhere from 10-15 books on the go at any one time so I can dip into whatever my mood is for that day. Some books I read start to finish without picking up another one and some books take me over a year. Lots of long airplane flights and boat trips help too!

  1. Memories of Ice: A Tale of the Malazan Book of the Fallen; Steven Erikson; 1187pp; Fantasy Fiction; January 3
  2. The Wine-Dark Sea; Patrick O’Brian; 295pp; Historical Fiction; January 5
  3. The Bone Tree; Greg Iles; 816pp; Fiction; January 10
  4. Indonesia, Etc.: Exploring the Improbable Nation; Elizebath Pisani; 416pp; Travel; January 22
  5. The Life of Elizabeth I; Alison Weir; 656pp; History; February 20
  6. Forbidden Places: Exploring our Abandoned Heritage; Sylvain Margaine; 256pp; Photography; March 1
  7. House of Chains: A Tale of the Malazan Book of the Fallen; Steven Erickson; 1040pp; Fantasy Fiction; March 27
  8. The Origins of Political Order: From Prehuman Times to the French Revolution; Francis Fukuyama; 608pp; Nonfiction; April 2
  9. Fresh Air Fiend: Travel Writings; Paul Theroux; 480pp; Travel; April 9
  10. Dirty Wars: The World Is a Battlefield; Jeremy Scahill; 681pp; Nonfiction; April 21
  11. Shift; Hugh Howey; 608pp; Science Fiction; May 10
  12. A Night Without Stars; Peter F. Hamilton; 640pp; Science Fiction; May 25
  13. Babylon’s Ashes; James S. A. Corey; 538pp; Science Fiction; June 27
  14. The Revenge of Analog: Real Things and Why They Matter; David Sax; 304pp; Nonfiction; July 11
  15. The Life-Changing Magic of Not Giving a F*ck; Sarah Knight; 224pp; Nonfiction; July 29
  16. The Bat; Jo Nesbo; 384pp; Fiction; August 18
  17. The Kingdom by the Sea: A Journey Around the Coast of Great Britain; Paul Theroux; 368pp; Travel; August 23
  18. Fast Food Nation: The Dark Side of the All-American Meal; Eric Schlosser; 362pp; Nonfiction; August 25
  19. The Commodore; Patrick O’Brian; 303pp; Historical Fiction; August 28
  20. Bleak House; Charles Dickens; 1088pp; Fiction; August 28
  21. The Yellow Admiral; Patrick O’Brian; 264pp; Historical Fiction; August 29
  22. What the Dog Saw: And Other Adventures; Malcolm Gladwell; 448pp; Nonfiction; September 3
  23. Crystal Fire: The Invention of the Transistor and the Birth of the Information Age; Riordan and Hoddeson; 368pp; History; September 3
  24. The Hundred Days; Patrick O’Brian; 272pp; Historical Fiction; September 5
  25. Blue at the Mizzen; Patrick O’Brian; 252pp; Historical Fiction; September 6
  26. Book 21 / The Complete Aubrey/Maturin Novels; Patrick O’Brian; 65pp; Historical Fiction; September 6
  27. Why Nations Fail: The Origins of Power, Prosperity, and Poverty; Acemoglu and Robinson; 544pp; Nonfiction; September 9
  28. Midnight Tides: A Tale of the Malazan Book of the Fallen; Steven Erikson; 960pp; Fantasy Fiction; September 15
  29. Dark Intelligence: Transformation Book One; Neal Asher; 402pp; Science Fiction; September 23
  30. The North Water; Ian McGuire; 272pp; Historical Fiction; September 26
  31. I Contain Multitudes: The Microbes Within Us and a Grander View of Life; Ed Yong; 357pp; Nonfiction; October 1
  32. Thirteen Days: A Memoir of the Cuban Missile Crisis; Robert F. Kennedy; 222pp; History; October 4
  33. The Peripheral; Willam Gibson; 485pp; Science Fiction; October 5
  34. Dust; Hugh Howey; 480pp; Science Fiction; October 10
  35. Night of Knives: A Novel of the Mazalan Empire; Ian C. Esslemont; 304pp; Fantasy Fiction; October 12
  36. Days Without End; Sebastian Barry; 272pp; Historical Fiction; October 13
  37. The Lighthouse Road; Peter Geye; 304pp; Historical Fiction; October 14
  38. Mr. Penumbra’s 24-Hour Bookstore; Robin Sloan; 288pp; Fiction; October 25
  39. Neutron Star; Larry Niven; 285pp; Science Fiction; October 25
  40. Archipelago: Portraits of Life in the World’s Most Remote Island Sanctuary; Middleton and Liittschwager; 264pp; Photography; October 26
  41. Atlas of Remote Islands; Judith Schalansky; 144pp; Nonfiction; October 27
  42. The Technician; Neal Asher; 503pp; Science Fiction; November 9
  43. Atlas of Cursed Places: A Travel Guide to Dangerous and Frightful Places; Olivier Le Carrer; 144pp; Nonfiction; November 15
  44. The Little Paris Bookshop; Nina George; 416pp; Fiction; November 18
  45. Platform Souls: The Trainspotter as 20th-Century Hero; Nicholas Whittaker; 288pp; Nonfiction; November 24
  46. Origin; Dan Brown; 461pp; Fiction; December 8
  47. Remarkable Books: The World’s Most Beautiful and Historic Works; DK; 256pp; Nonfiction; December 25
  48. Mrs Rosie and the Priest; Giovanni Boccaccio; 54pp; Fiction; December 27

Live online training from Kimberly in early January!

We’ve been considering doing some live, online training classes for a while now, and with the demand for Kimberly to teach an Immersion Event on partitioning, we’ve decided to take the plunge and present our first live, online class in January!

Kimberly will be teaching our new IEVLT: Immersion Event on Very Large Tables: Optimizing Performance and Availability through Partitioning live, online over January 9-11, 2018.

Rather than have people try to watch a full day of training at their computer for one of more days, the class will run from 10am to 3pm PST each day, with two 90-minute teaching sessions, each followed by Q&A, and a lunch break. We chose to do this, and to spread the class over a few days, so the times work well for those in the Americas, Africa, and Europe (we might consider doing some time-shifted classes for those in Asia and Australia/New Zealand, depending on demand). We also realize that this is complex, architectural content, so want to give attendees time to digest each day’s material, plus extensive Q&A.

The modules covered will be:

  • Horizontal Partitioning Strategies
  • Partitioned Views
  • Partitioned Tables
  • Tables, Indexes, Keys, and Relationships
  • Implementing the Sliding Window Scenario
  • Key Partitioning Concerns and Considerations
  • Partitioning Techniques Combined
  • Review

The price of the class is US$595 and you can get all the details here.

The class is already half full, as we announced it in our newsletter on Monday. All future live, online training classes will always be announced first in the newsletter, with a discount price available for 48 hours before the class is announced here and linked into the training section of our website, at the regular price.

We decided to start teaching some live, online classes as we recognize that not everyone can travel to our in-person classes, or take that time away from work or family, or simply have travel budget as well as training budget. People also have different ways they learn, some preferring in-person training, some preferring recorded, online training, and some preferring live, online training.

We’ll be doing more of this in 2018, on a variety of topics, so stay tuned for updates (and discounts through the newsletter).

We hope you can join us!

New class: Practical Machine Learning

We’ve added a third brand-new class to our Spring line-up in Chicago: Immersion Event on Practical Machine Learning.

It’s a combination 2-day class plus 3-day class, taught by industry expert Rafal Lukawiecki, presented on May 7-8 and 9-11, 2018.

This course has two parts. You start with the 2-day Part 1: Introduction to Machine Learning, AI & Data Science with Azure ML and SQL Server followed by the 3-day Part 2: Intermediate Machine Learning in R on SQL Server and Microsoft ML Server. The first part introduces the most important concepts and tools, while the second part teaches you R and how to use it for machine learning on the Microsoft platform.

Target Audience

  • Part 1: Analysts, budding data scientists, database and BI developers, programmers, power users, DBAs, predictive modelers, forecasters, consultants.
  • Part 2: Data scientists and attendees of Part 1

If you’ve attended a prior course on Machine Learning, like Rafal’s week-long class Practical Data Science that was offered in 2015–2017, and if you’re versed in model validity, accuracy, and reliability, consider attending Part 2 only. Ask yourself these questions: can I explain the difference between cross-validation and hold-out testing, do I know which business metrics correspond to precision and which to recall, is model accuracy more important than reliability, and how does a boosted decision tree work. If in doubt, please attend both Parts 1 and 2.

The modules are as follows:

Part 1: Introduction to Machine Learning, AI & Data Science with Azure ML and SQL Server (Mon-Tue)

  • Machine Learning Fundamentals
  • Algorithms
  • Data
  • Process of Data Science
  • Introduction to Model Building
  • Introduction to Model Validation

Part 2: Intermediate Machine Learning in R on SQL Server and Microsoft ML Server (Wed-Fri)

  • Working with R
  • Data Preparation in R
  • Plots and Visualizations in R
  • Clustering, Segmentation, Anomaly Detection
  • Classification
  • Classifier Validation
  • Regressions
  • Regression Validation
  • Deployment to Production

You can read a more detailed curriculum here and all the class registration and logistical details are here.

We hope to see you there!