PASS Summit 2015: Day 2

8:20 AM

We’re off and running with Adam Jorgensen, PASS EVP of Finance.  Adam takes this opportunity to provide an update about the financial status of PASS as this satisfies the requirements of the by-laws.  The largest source of revenue is the PASS Summit (not a surprise), bringing in just over 7 million dollars (of the 8 million generated in the 2015 fiscal year).  The finances continue trend upward, which is great.  Finances support the community through events all year long.  This year, 78% of every dollar taken in goes back to a community program.  PASS is in great financial health, increased reserves to 1.14 million dollars.  Starting this year, portfolio-level budget summaries will be published, to make the process more transparent to the community.  Last year goals for 2015 were to focus on support for SQLSaturdays and Chapters, among others.  PASS Summit will be in Seattle through 2019.  SQLSaturday website was relaunched this past year to help better support the events.  This year, goals include the BA Community Portfolio, refocus investments to community profiles, global growth program, sales portfolio, technology investments (including a re-design of ELS: this makes Jes happy).  Adam wraps up by thanking Amy Lewis, outgoing board member.

8:33 AM

Adam finishes up and EVP Denise McInerney comes on stage.  Denise takes a minute to thank Bill Graziano, who is the outgoing Immediate Past President.  Bill has been a member of the board for 10 years.  ELS: I’m personally a big fan of Bill, I worked with him on the NomCom.

Denise moves on to the PASSion Award.  There were 71 Outstanding Volunteers this past year.  This year’s PASSion Award goes to Lance Harra.  He runs the Kansas City SQLSaturday and was an integral part of the program committee.  If you are interested in becoming a part of the SQL Server leadership team, stop by the Community Zone this week.  There are always ways to get involved with PASS.

There are over 150,000 members of PASS.  There are 3000 people from over 95 countries tuning in live.  Yesterday PASS introduced foundation sessions, which were offered by Microsoft (four of them yesterday).  Over the years PASS has grown its offerings to meet its members needs – virtual chapters, 24 Hours of PASS, SQLSaturday, user groups, and more.

Today is the Women in Technology lunch (11:45) sponsored by SQLSentry, and the keynote speaker is Angie Chang.  It will be live streamed on PASSTV.  Today is the Board Q&A at 3:30.  Tonight is the Community Appreciation Party at the EMP Museum at 7 PM.

PASS Summit next year is scheduled for October 25 – October 28 – early bird pricing is available!

Today’s keynote speakers are Dr. David DeWitt and Dr. Rimma Nehme. (ELS: TWO OF MY FAVORITES!!!)  They are both at the Microsoft Jim Gray Systems Lab in Madison.  Data Management for the Internet of Things.

8:45 AM

Dr. Nehme takes the stage.  She mentions that it’s harder to present a keynote together than individually.  She will start, Dr. DeWitt will come in, then Dr. Nehma will wrap up (dessert!).  What, why, how and of IOT.

Disclaimer: not announcing a product.  Goal is to inform, educate, and inspire (and entertain a bit).

Wants to begin with a new reality.  Things around us have a voice that can communicate to us.  IOT is a collection of devices and services that work together to do something useful.  Basic formula: take a basic object, add controller, sensor and actuator, add the internet, and then you get the internet of things.

Take the sensors and actuators, add connectivity and big data analytics, and then you can provide new services and optimization.  The target is to create value (make money).  What does that typically look like?  Collect data from sensors, aggregate it, analyze, then act on it. This is a continuous loop.  There are 2 types of IOT that people agree upon.  On one side have a consumer internet of things – things that are wearable, related to us as humans (phone, watch, etc.) then have things that are industrial (cars, factories, etc.).

Consumer IOT: fitbut, Nest, Lumo.  What can they reveal about us?  Health info, house information, driving habits.  You can analyze that information and make predications/revelations.  The Industrial Internet of Things (IOT) can be connected, and then significant value can be realized, particularly in Industry.  It is still in its infancy.  There are four types of IOT capabilities: Monitoring, Control, Optimization, Autonomy.  The analogy of this to human development..  We are in the “terrible twos” of the IOT development.  Why IOT?  We are at the peak of the hype right now (based on Gartner).  There is a growth of “things” connected to the internet.  In 2003 had about 500 million devices connected to the internet.  Have 12.5 billion by 2010.  Around 2008, the number of things connected to the internet exceeded the number of people.  In 2015, at 25 billion things connected to the internet.  The value to customers is huge.  The power of 1% – if you can improve 1% in fuel savings in an industry like aviation, health care, or power generation, that’s extremely significant.

Why is this happening now?  More mobile devices, better sensors and activators, and BI analysis.

For IOT How?, Dr. DeWitt comes on stage.  Dr. DeWitt is going to talk about the services available.  There are a lot of challenges – a large number (and variety) of sensors.  There are A LOT of devices sending data.  Sensors are frequently dirty, and it’s hard to distinguish between dirty readings and anomalies.  And then there is just the volume of data that’s being sent into the cloud.  One of the biggest challenges is device security.  How do you prevent them from overwhelming cloud infrastructure or impersonating a device?  And then there’s cloud-to-device messaging.  Sometimes the device is not online.  Therefore the device may miss a message, so persistent queue and reliability is needed.  How do you deploy this and get the IOT set up?  We’re not going to tackle that today.

There are differences between consumer and industrial IOT.  In consumer IOT have to worry about battery and power failure, more cost-sensitive, and might be a simple embedded device, or it could be a powerful sensor, and finally, consumers have wireless (industrial has unlimited power, full-fledged, wired, and depends on needed functionality).  Rest of talk will focus on industrial.  Note: one size fits none.

Today’s IOT: Just Do It Yourself.  The state of the art is still rather primitive.  What are the ingredients that go into IOT?  The basic block diagram, out in the field you have devices with a sensor and actuator (e.g. sense temp, humidity, in a Nest thermostat).  Up in the cloud, have event/data aggregator.  Device to Cloud (D2C) is how the data gets from the device up to the cloud.  You can feed this data into an application, into event/data storage, into a real-time processing engine (real time), and that *can* use a device controller and send it back to the device (C2D = Cloud to Device).  Azure IOT services exist.  Two main components: Azure Iot Hubs and Azure Event Hubs.  The data management is done through Azure Stream Analytics, DocumentDB, SQL Azure and SQL-DW, Azure HDInsight and Azure Machine Learning.  and then use PowerBI and Excel to visualize the data.

Azure IOT Hub (an Azure PaaS Service), this is the cornerstone of IOT.  It receives events and routes them.  It is scalable to millions of devicees, and it provides per-device instance authentication.  It can send commands back to the devices.  Within the hug, every device has it’s own send endpoint, to which the sensors will send events.  On the output side, is a set of partitions, into which data gets routed.  The number of partitions is created when the service is created in the cloud.  A hash function routes it to a partition.  Event consumers then “pull” events from the Receive EndPoint.  There is a C2D Send Endpoint that can send messages out, and then get routed to a message queue that guarantee once delivery out to the device’s actuator.

One thing you can do with events is pull them out of the IOT HUB and they go to the Event Consumer such as SQL Azure (doesn’t have a nicer sexy symbol like SQL Server), into HDFS, into Azure Storage, or into DocDB (these are examples).  Analyzing the events, then, can be done via SQL Server, or use SQL-DW and Polybase, Hadoop from HDFS (or Hvie/Storm), or DocDB.  All of these are great opportunities to store events.  A neat thing to do with IOT data is LEARN from it (e.g. when the boiler might explode).

Options for real-time query engine include Azure Stream Analytics or Apache Storm on HDInsight.  What’s a real-time query engine?  Traditional RDBMS with data on disk, send in a query, get data back.  In Dr. DeWitt’s mind, the real-time streaming is taking a sequence of events, and some queries that will operate over those events, and the query will find IDs of boilers that are about ready to explode based on PSi.  As query processes stream events, it will eventually produce results.  Can have multiple queries operating over the same set of events, or different streams.  Dr DeWitt encourages us to learn about stream analytics.

There is no data stored, the queries are just continually running, data flows through the query, outputs results.  When you see something important, what do you do?  Send a message to IOT hub to do an action (e..g open pressure release valve).  Field gateway – Raspberry Pi, running Windows 10, has WiFi – that’s a field gateway.  There are two primary use cases: when a sensor/device cannot itself connect to the internet, or for complex objects (e.g. smart cars) with multiple sensors/actuators.  Two flavors: opaque (only field gateway has identity in IOT hub) and transparent (each device is registered in IOT hub.  The field gateway are processors with memory and processors.

How to manage IOT metadata?  per-device metadata is not stored in a database system at present time so no query support.

Device security is super critical for IOT deployment.  Devices must have unique identities, and must PULL to obtain C2D commands (no ports open to reduce attacks).  Main takeway: it is PUSH to the cloud.  All the IOT events get pushed up into the cloud.  It was a good first effort.  But what are the problems with pushing everything to the cloud? Not enough bandwidth, requires connectivity, latency, data deluge (from boring sensor readings), storage constraints (storing EVERY event), speed, main point: wastes network bandwidth, computational resources, storage capacity and bandwidth processing for NON-INTERESTING events.

Go back to boiler example…Running the same query over and over, waste bandwidth sending the reading every second.  Centralizing all data from multiple systems might overload the system.  Here is their insight: exploit the capability of the field gateway.  It can do local processing and control.  Have the boiler with sensor and actuator.  Then you have a field gateway, and in that, going to run a streaming database system, and install on that boiler gateway control program, and run data through.  If run streaming engine there, can run any number of queries, might send average pressure reading for 60 seconds of data up to the IOT.  This is a better approach – reduce what pushing up to the cloud, and what needs to be stored.

How can we do better?  Dr. Nehme comes back on stage…  (she has changed her outfit…but don’t tweet about it…she’s a jeans and tshirt girl (I KNEW IT)

Fog computing – all about computing on the edge.  It is not cloud vs. fog, it is cloud + fog.

What’s the fog?  It’s like “predicate pushdown”.  Never move the data to the computation, move the computation to the data.  Devices perform some data pre-processing and compression, the cloud is a big gorilla that can do the management, processing, and machine learning.  How can we do better?  Real-time response, scalability, metadata management, GeoDR of IOT hubs.  IOT is a database problem, not just a networking problem.  It hasn’t been database-centric before, but trying to address that.

Want to take existing IOT Azure services and expand on them.  Proposing Polybase for IOT (not a product announcement, just an idea).  What is vision? Declarative language, complex object modeling, scale able metadata management, discrete and continuous queries, multi-purpose querying, computation pushdown.

Declarative language: if dealing with IOT, only choice is to use imperative language.  Have to explicitly specify how you want to see something.  What about IOT-SQL?  A declarative language where you can select information from the sensors.  If have tables specified as buildings, room, temperature sensors, etc.  With temperature sensors, have columns that looks like regular database.  Need to figure out how to model complex objects – for example, a room on a floor in a building, – need a model for this.  Have a notion of a shell database – it is a regular database that stores metadata, statistics, and access privileges – can perform authentication, authorization and query optimization against that database.  As far as these processes are concerned, they don’t need the actual data.  Now expand this to the devices.  The IOT shell also gives a simple abstraction for sensors, actuators, and distributors.  The shell can be stored in SQL Azure, DocDB, etc.  It’s JUST a database.

What about querying devices?  One query is ExecuteOnce: push select to device, it sends results, we’re done.  ExecuteForever, push SELECT to device, then the device continually sends results back to client.  When done, send signal we’re done and query stops running.  Then have ExecuteAction: send a SELECT and then an action, and the action gets fired when predicate is met.  Can do execution once, or forever.

Back to temperature sensor table…need some delcarative queries.  ExecuteOnce – get the count of all hot locations.  The optimized plan is generated, data is moved, and then work is done up in the cloud.  Not a lot of pushdown here.  ExecuteForever query – record all hot locations up in the cloud, and execute forever, the optimizer might produce a different plan (does some partial aggregation before pushing data up into the cloud – larger computation is done in the “fog”).

ExecuteAction: turn on AC in all the hot locations.  Larger computation and the action is pushed down in the fog, and only interesting events are pushed up into the cloud.  Multi-purpose query – based on results, some could go to one location, some could go to another location.

The Polybase for IOT Wrapup – use SQL front end with Polybase for sensor/actuator metadata management and querying.  Exploit Polybase’s external attribute mechanism to allow SQL queries to reference sensor values…and then one more thing I didn’t get :)

Why should we, as data professionals, care?  When a new technology rolls over you, you’re either part of the steamroller or part of the road (didn’t get the attribute).  Key takeway: the amount of data to manage is exponentially going up.  Need to step back to see what success looks like.

Dr. Nehme has announced that this is their last keynote.  Why? Dr. DeWitt…they have done 7 of these.  There are a lot of great speakers at MS, and he is sure there are people who are better speakers.  Dr. DeWitt and Dr. Rehme are “parting ways”.  She is finishing up her MBA and moving on.  Dr. DeWitt is starting to think about retirement.  After 40 years thinks it’s about time to give up the full time gig.  In 10 years…  Have not seen the last of Dr. Rehma – whether it’s at Microsoft or at a competitive.  Dr. DeWitt says this has been one of his brightest spots in his career.  He says it’s been a terrific experience.  He will think about this community for many years to come.  (ELS: I admit, I’m a little teary.)

PASS Summit 2015: Day 1

Well friends, here I sit again, at the blogger’s table, ready to kick of Day 1 of the PASS Summit here in Seattle, Washington.  My trusty side-kick, Perry, is with me as usual, and we’re joined by Bunny this year as my 8 year old insisted I am bring them both. Who I am to argue with her?

Some notes about today

It’s National Chocolate Day!  I plan to celebrate all day :)

I present twice today!  My first session is right after the keynote: Kicking and Screaming: Replacing Profiler with Extended Events in Room 6A from 10:15 to 11:30 AM.  Note that this session was a little later in a different room than originally scheduled.  My second session, Statistics and Query Plans, is from 3:15 PM to 4:30 in 6B.  I hope to see you at one of my sessions, feel free to come up and say if we haven’t met before (or if we have!).

Today’s keynote, Accelerating Your Business With a Modern Data Strategy, is headlined by Joseph Sirosh who is a Corporate Vice President in the Data Group at Microsoft.  And we’re off…

8:21 AM

Up first today is PASS President Tom LaRock.  This is Tom’s last year as President, he’ll next step into the role of Immediate Past President.  He mentions #SQLFamily and says that everyone is free to give him a hug.  That could be a lot of hugs.

Attendees from over 58 countries…over 2000 companies are represented, and the Microsoft team will be everywhere this week – stop by the SQLClinic if you have any questions you need help with.

For those of you not here, please follow along on  PASS TV, just head over to the main page for Live Streaming.

Tom introduced the PASS Board of Directors and encouraged members of the community to talk to the board this week to help them understand how to serve the community better.  Tom mentions the Board Q&A on Thursday at 3:30 PM in 307-308.

There are 5,500 total registrations this year for Summit (note: that’s not individuals…if you register for a pre-con and the conference, I think that’s 2 registrations, not 1).  Tom asks for a show of hands from newcomers…there are a lot.  ELS: Those of you who are here for the first time, try to meet people!  If you’re an introvert, I know that’s hard, but take a risk!  Say hi, find something in common!

The SQL community is the gold standard for technical communities.  ELS: I don’t disagree, I have friends in other technical disciplines, and they have nothing like what we have.

There are over 200 sessions and workshops this week.  Use the mobile app Guidebook to stay on top of any schedule changes.  On Twitter follow along with hashtags #sqlpass and #summit15.

The Birds of a Feather lunch will take place during lunch on Friday where you can talk to people with an interest in a specific feature/area.

Don’t forget out Sponsors who make this entire event possible.  There are some fantastic companies that support the SQL Server community.  PLEASE make time to go talk to them this week.  The Exhibitor Reception is tonight, after regular sessions end.

Tom closes by saying how proud he is to be a member of the #SQLFamily community.  He’s been a member since 2004.  I think he’s getting a little choked up.  Oh.  HUGS TOM!

 8:37 AM

Joseph Sirosh takes the stage.

We live in an age of data.  The ability to extract that data and use it is changing our daily lives.  All of the worlds data was analog 30+ years.  Then we got DVDs and such which started to digitize data.  When the internet came along, data suddenly had an IP address.  Connected data can be moved around and joined with other connected data.  Which means you extract intelligence from it.  The vast majority of today’s data is digital.  Much of that in the cloud.  Fast forward to 2020, there will be 50 million petabytes of data, mostly in the cloud.  Fifty years ago, hardware drove new customer experiences.  Then came the age of software.  Digitizing everything.

In the new world, data will predict everything.  We can use this data to develop models so that when, for example, people come in to the ER with a problem, you can put in data collected and use a model to determine a path of care.

Joseph brings up Eric Fleischman who is the Chief Architect and VP of Platform Engineering at DocuSign (we use their site!), they chose to use SQL Server because they believed Microsoft would be there for them (and they have been).  They made an investment into the telemetry of the system that process millions of data points about the performance of the actual system.  That system is scaling literally to the OLTP system.  There are some improvements in the HA/DR stack for them in 2016, along with the encrypted features.

SQL Server 2016 is meant to be the all engines of data that you can build your data on – both in house and in the cloud.  Innovate first in the cloud with an accelerated speed (push new code once a week).  The pain in this system translates into changes in software very quickly.  When you build and operate in the cloud, you take innovation and bring it back to packaged software (SQL Server 2016).  Companies like Oracle cannot claim that…who build locally and then ship to the cloud.  Amazon will state they are only in the cloud, it’s a cloud-only feature. But “we” know better.  Feet on the ground and head in the cloud.  You have to build products to operate both in the sky and the ground.  Microsoft is the only company to do that.  This community is making Microsoft number 1 in the age of data.


There’s a video with some feedback from fellow MVPs about SQL Server 2016…  Joseph turns the stage over to Shawn Bice (General Manager, Database Systems Group).  Haven’t shipped 2016 yet, but it powers everything in the cloud.

Seven big bets…all of these are built-in.

From OLTP perspective – SQL Server is recognized as a leader.

SQL Server is the most secure database.  SQL runs some of the most scalable data warehouses in the world.  Mobile BI is built into SQL Server, it’s about that mobile workforce and getting visualizations to them.

First big bet: HA/DR.  Have learned a lot from partnership with DocuSign.  DocuSign is using some of the fastest IO subsystems with FusionIO.  Have A LOT of data moving across the wire to secondaries.  Have done a ton of work with algorithms to improve updates.  Have customers that use Azure along with on premise all the time.  You can enroll an Azure DB with an on-prem system to create a DR site.  For all of you that have used DB Mirroring: want to use an AG but can’t domain join it.  In SQL 16, can stand up HA environment, don’t have to domain join anything.  Woohoo!  Introduce load balancing around read scale, so don’t have to point clients to every secondary.  The stack for on prem is the same that’s in Azure, and they do failovers every day.  They had a data center that was on fire and failed over every customer in China in about 5 hours.

Ok, I need to get to my first session, I’ll be back tomorrow!  Have a great day!


Costumes, PASS Summit, and Argenis Without Borders 2.0

I’ve never been a fan of Halloween. Obviously not because I don’t like candy or chocolate (you all know I like sweets right?).  It’s because I dread the costume. This wasn’t a problem as a kid. My costumes included, in no particular order…

  • A mummy (though I got tangled in a couple trees due to my bandages)
  • A clown (Tim Chapman take note)
  • A crayon (not a lot of flexibility in that costume)
  • R2D2 (I was a nerd even when I was young)
  • Princess Leia (from Episode IV, not the bikini costume)
  • A robot (hard to walk through doors while dressed in a box)

Those were fun times. Then I got to college and Halloween took on a whole new meaning. I am not one to dress up in any tight-fitting barely-there costume, so I resorted to a farmer one year (insert Iowa joke here) and other years just avoided going out all together. Finding something cool was tricky. My friend Jori was awesome at costumes, she still is (last year she, her husband, and their 1 ½ year old went as the Three Amigos…genius). But I digress.

So this year, when Argenis Fernandez asked if I would participate in his second fundraiser for Doctors Without Borders and wear a costume if he raised at least $5000, I of course said yes…knowing full well I would have to find a costume.

Enter my friend Jes.

Yes Jes, yes it is...

Yes Jes, yes it is…

I now have a kick @$$ costume if I do say so myself, and you’ll see it at the PASS Summit (I’m speaking on Wednesday at 4:45 PM, if you’re interested!). In the meantime, if you haven’t already, head on over to Argenis’ page and donate to a fantastic cause.   The amount doesn’t matter…$5, $10, $20…whatever. It all adds up, it all helps. Every little bit.  Thanks for considering, and I hope to see you at Summit!

In Support of PASS Board of Directors Nominee Ryan Adams

Reminder: Opinions are my own.

There are four candidates running for three Board of Director positions this year (in alpha order):

  • Ryan Adams
  • Argenis Ferndandez
  • Tim Ford
  • Jen Stirrup

You can read more about each candidate through links on PASS’ Elections page here. I do know all four candidates personally – some a bit better than others – and I know that they are all great individuals who have all contributed in numerous ways to the SQL Server Community over the years.

But this post is specifically about Ryan Adams and why he’ll be getting my vote for a Board of Directors position. I support Ryan because I don’t just know him, I have worked with him. I had two opportunities to work closely with Ryan in the last three years on PASS-related teams.

I first got to know Ryan in late 2011 when I took over the PASS Performance Virtual Chapter. I put out a call for volunteers in November 2011 to help run the team. Ryan applied with the intention of managing the web site , but I recognized that he came to the group with a lot of experience already (at that time he was on the board for the North Texas User Group, he was a Regional Mentor, and he was helping to organize SQLRally), and I needed to leverage that. I convinced him to handle marketing for the group, and though we didn’t know each other well in the beginning, our small team came together through regular conference calls. We all had different responsibilities, and we were off and running. We created (in my opinion) a very solid VC over the course of the year. Ryan was an integral part of that success. He created his own goals and figured out how to meet them. He set up new avenues for marketing the VC meetings, and came up with some great ideas along the way, including the Performance Palooza. We held the first one in December of 2012, and the Palooza has continued to this day for the Performance VC (its attendance this year grew 175% compared to last year).

At the end of 2012 I stepped down from running the team, as I had only committed to a year, and I also felt it was good for someone to have another opportunity in a leadership position. Ryan stepped forward without hesitation; he was a natural fit. This was confirmed when we finally met in person at the 2012 PASS Summit for a dinner to celebrate our year, and Ryan was already thinking ahead to 2013 and making plans. Ryan has continued to run that group and it has continued to grow over the years, now averaging 200-300 attendees per session.

I again worked with Ryan in 2013 when we were both members of the PASS Nomination Committee. I was again impressed. The NomCom requires its members to review applications by a specific date, and also attend meetings and candidate interviews. Collectively, we all tried to meet deadlines and attend every meeting – I’m not sure that Ryan missed any. During our candidate interviews Ryan asked thoughtful questions, and provided great insight and evaluation during our follow up discussions. Post-nominations, he gave critical consideration to the process that we went through provided both big-picture and detailed analysis of it, contributing to the changes that occurred for our nominating committee and the next.

In the last three years I have watched Ryan continue to contribute to the community through his volunteer responsibilities and his speaking. He’s become a solid speaker, presenting at many SQL Saturdays and the PASS Summit. He is comfortable in front of large and small groups and has excellent written and verbal communication skills. He has in-depth knowledge about PASS and how it works, which he’s been accruing since he started helping out the North Texas User Group all those years ago. Ryan has the ability to take on projects and responsibilities, take action, and see them through the end. He gets thing done. He has a positive attitude and he’s a critical thinker. I know Ryan would be a solid member of the PASS Board of Directors, and it’s been a privilege to watch him grow as a leader within our community.

Best of luck to all the candidates in this year’s election, and I encourage every voting PASS member to take time to learn about each candidate so you can vote based on data.  Read through each candidate’s page on the PASS site, and if you still have questions, visit the election forum and/or attend the Town Hall and Twitter chat events.  The board makes a lot of decisions that affect the SQL Server community – it’s a good idea to know who’s representing you.

Don’t Hard Boil Your Next SQL Server Upgrade

SQL Server upgrades and hard-boiled eggs. What do these two things have in common? Probably nothing, but last week Jon and I were having a conversation about an upgrade I need to do for one of our clients. Now I can’t go into specifics because of the NDA, but I can give you some of the pieces:

It’s a two-node cluster, there are two instances of an older version of SQL Server Standard Edition, they want to upgrade to a recent release of SQL Server, there’s replication involved…and we may or may not reuse the existing servers.

Now, I like talking through different options for this kind of challenge with someone else on the team, and Jon loves this type of thing. I love solving problems, it’s one of my favorite parts of my job. I think that Jon thrives on it. We’ll discuss some project or process and I will have an idea for how to do it, and then three hours later I’ll get another email or a phone call from Jon that starts with, “Well you could also…”

Our conversation about upgrading was interspersed with a conversation about getting into better shape and eating better (yes I’m usually going in three directions at once, aren’t we all?), and Jon mentioned hard-boiled eggs because they are a good breakfast protein. I asked him if he was still buying them already boiled…because he used to do that. He said no (hooray!), that one of his friends now boils them for him in big batches (nice friend!). Then he mentions how difficult it is to boil eggs.

And I’m thinking, “Are you kidding me?” He’s one of the smartest people I know and boiling eggs is a challenge? So I sent him the steps below, which is this which is an adaptation of instructions provided in Cook’s Illustrated (great cooking magazine if you’re interested):

  1. Place eggs in a large pan and enough cold water so that it’s at least 1” above the eggs. Add a splash of vinegar.
  2. Place on burner and set to high.
  3. When the water starts to boil, set a timer for 5 minutes.
  4. When the timer goes off, turn off the burner, set another timer for 7 minutes, let the eggs sit in the hot water.
  5. When the timer goes off the second time, drain the hot water, fill with cold water and ice.
  6. Let sit for a couple minutes, then either drain and put in the fridge, or peel and eat.

And Jon replies with this (yes I know the image is large, it’s so you don’t have to click on it to read the text! :)

Upgrading is easier than boiling eggs

Upgrading is easier than boiling eggs

I laughed. A lot. I was thinking, “I’ll take boiling eggs over an upgrade any day, no matter how much I love SQL Server.”

And then yesterday I decided to boil a couple eggs for breakfast. I put them in the pan with plenty of water and a splash of vinegar, turned on the burner, and then went to my office (just 20 feet away) to work on my Insider Video. Forty minutes later (yes, 40) I remembered the eggs. I forgot to set a timer. I always set a timer, but yesterday I just forgot. I ran to the kitchen…there was about an inch of water left in the pan. I was lucky. Nothing destroyed, the eggs were even edible! But here’s the thing…

It doesn’t matter how many times you’ve done something before, and how well you know the steps involved and KNOW that they work. Sometimes unexpected events happen or a single step gets missed that causes a problem (or a disaster).

Need a real-life example? Jon had a customer that wanted to remove partitioning for a huge table. They had gone through the steps many times in a Test environment, and they were all documented. When it came time to run the process in Production, however, the step to disable the nonclustered indexes on that table before removing partitioning was missed. As a result, the nonclustered indexes (on that extremely large table) were rebuilt three separate times. The step was in the documentation, and had been done during the tests, but for some reason wasn’t done during the process in Production.

So back to that upgrade…we’re still planning for it. If I get approval from the customer, I’ll share the details and let you know how it goes. Until then, keep making those checklists and testing your upgrades. You just never know what’s going to happen.


Today I opened up a SQL Server ERRORLOG and saw these two messages repeated every 20 seconds or so:

Starting up database ‘AdventureWorks2014′.

CHECKDB for database ‘AdventureWorks2014′ finished without errors on 2015-08-23 02:15:08.070 (local time).  This is an information message only; no user action required.

When you initially see these two messages repeated over and over, it might seem like SQL Server is caught in some issue with recovery.  Or you might think it’s running CHECKDB over and over.  Neither are true.  The database has AUTO_CLOSE enabled.  (And you see the CHECKDB message because it’s reading the boot page and noting the last time CHECKDB ran successfully…to see what updates that entry, check out my post What DBCC Checks Update dbccLastKnownGood?)

When AUTO_CLOSE is enabled, after the last user exits the database, the database shuts down and its resources are freed.  When someone tries to access the database again, the database reopens.  You might be thinking that for databases that are not accessed that often, this might be a good thing.  After all, freeing resources and giving them back to SQL Server for use elsewhere sounds useful.  Not so much.  There’s a cost associated with that shut down, and a cost to open the database back up when a user connects.  For example – shutting down a database removes all plans for that database from cache.  The next time a user runs a query, it will have to be compiled.  If the user disconnects, the plan is freed from cache.  If someone connects one minute later and runs the same query, it has be compiled again.  You get the point: this is inefficient.  And really, how many databases in your production environment do you really not access?  If you’re not accessing the database, why is it in a production instance?  If you want a few more details on AUTO_CLOSE, check out the entry for ALTER DATABASE in Books Online.

I am sure (maybe?) that there are valid cases for having AUTO_CLOSE enabled.  But I haven’t found one yet :)

On top of the resource use, realize that every time the database starts up, you’re going to get the above two messages in the ERRORLOG.  In the log I was looking at, there were multiple databases with this option enabled, so the log was flooded with these messages.  In general, I’m a huge fan of cycling the ERRORLOG on a regular basis (just set up an Agent job that runs sp_cycle_errorlog every week), and I try to reduce “clutter” in the log as much as possible.  This means don’t enable a setting like AUTO_CLOSE which can put in all those messages, and use trace flag 3226 to stop logging successful backup messages (they still go to msdb).

Oh yes, to disable AUTO_CLOSE:




Backup checksum default option in SQL Server 2014

The SQL Server team snuck in a new server configuration option in the 2014 release (I bet thanks to this Connect item even though it’s still Active), and it’s not documented so I just stumbled upon it recently.  If you run:

SELECT * FROM [sys].[configurations] ORDER BY [name];

you’ll see that there are 70 rows in the output (in 2012 there were 69) and the new one is:

backup checksum default

The option is disabled (set to 0) by default.  To enable it, simply run:

EXEC sp_configure ‘backup checksum default’, 1; GO RECONFIGURE WITH OVERRIDE; GO

As a quick reminder, adding the CHECKSUM syntax to the backup command forces SQL Server to verify any existing page checksums as it reads pages for the backup, and it calculates a checksum over the entire backup.  Remember that this does not replace CHECKDB (check out Paul’s post – A SQL Server DBA myth a day: (27/30) use BACKUP WITH CHECKSUM to replace DBCC CHECKDB – for more details).  So what does this server option do?  Well, since it’s not yet documented (I filed a Connect item here) I did some testing to confirm what I was expecting.

Within Management Studio I ran a simple backup statement:

BACKUP DATABASE [AdventureWorks2014] TO  DISK = N’C:\Backups\AdventureWorks2014_checksumtest.bak';

Then I checked the output in msdb:

SELECT [bs].[database_name], [bs].[backup_start_date], [bs].[backup_finish_date], [bs].[has_backup_checksums], [bs].[user_name], [bm].[physical_device_name] FROM [msdb]..[backupset] [bs] JOIN [msdb]..[backupmediafamily] [bm] on [bs].[media_set_id] = [bm].[media_set_id];

Backup information from msdb

Backup information from msdb


Check it out…with the server option enabled, I don’t have to include the CHECKSUM syntax to have SQL Server perform the backup checksum.

If you’re running SQL Server 2014, I highly recommend enabling this option for your instances, and if you rely on Glenn’s scripts for instance reviews, I know he’s adding a note about this to the next set :)

PASS Summit 2014: WIT Lunch

Two posts from me in one day?  What’s up with that?!  Well, today at the PASS Summit we also have the WIT luncheon, where Kimberly Bryant, who is the founder of Black Girls CODE, will be speaking.  I am live-blogging this event as well, so watch this post for updates starting around 12:15PM EST.  If you want to learn more about Black Girls CODE, check out this MSNBC interview.


Denise McInerney is introduced first – she asks how many people were at the first WIT lunch, back in 2002 (I am pretty sure Denise has been a part of WIT since its inception – a long-time leader within the community).  The WIT luncheon has grown a lot in the past 10+ years – today’s lunch has over 900 attendees.  Denise brings out Kimberly Bryant – such a different setting this year, just Denise and Kimberly on stage.  Denise is going to ask a few questions, then open it up to the audience and people watching on PASStv – you can tweet your questions and include the #passwit hash tag.

Black Girls CODE is a non-profit organization started in the Bay Area in 2011.  What really drove Kimberly to make a change when she recognized that her daughter, who was 12 at the time, might be following in her footsteps.  Never thought her daughter was an engineer.  But she was a heavy gamer (World of Warcraft, D&D) and spent a lot of time on the computer.  Her daughter was at the age where she could learn and create with a computer – and that was a life-changing moment for her.  Her daughter first wanted to grow up and be a game tester :)  Once she went up to a programming camp, she saw that the environment actually allowed her to create, not just be a participant.  As a parent, Kimberly noticed that she was only one of three girls at the summer camp, and the only person of color at the camp (out of about 40 campers total).  At that point, Kimberly knew she had to make a difference, not just for her daughter, but for other daughters.

Question from Denise: “Why is still so hard to get girls and young women interested in technology?”  Kimberly cites a Girl Scout study that showed tha tif you surveyed girls BEFORE they get to middle school, over half the girls show an interest in STEM, but by the time they get to high school it’s less than 5%.  In some cases, girls don’t have support from parents and teachers.  There are fewer opportunities for girls to flex the STEM skills.  Kimberly says she hates the pink aisle.  Do Legos need to be pink?  (ES: They don’t, I grew up without pink or purple Legos and played with them all the time.)

What type of programs does Black Girls CODE run?  The secret sauce is the environment of girls in the environment to do coding and engineering and they have relatable leadership – the women that come in to teach the programs.  Over 75-80% of the instructors are women.  They are reflections of what the girls can become, and that gives the girls the ability to see the possibility.  Kimberly had a counselor who said, ‘You’re good in math and science, you should go into engineering.”  Kimberly didn’t know what that looked like – what does an engineer do, what do they look like?  But if you’re able to actually see that, suddenly you have an idea of what you can really do.

Denise asked what languages are taught via Black Girls CODE?  Kimberly explained that in the beginning they didn’t know what the girls would be willing to learn, so it was open in the beginning.  The goal was to always teach them Ruby – and Kimberly had a core team that knew Ruby.  Also did some testing with Python, but do a lot with open source learning.  Have also started to talk to organizations about coding – she has talked with Lynn Langit, and Lynn’s program (Teaching Kids Programming) teaches Java.

Question from Denise: How can people who want to bring STEM education to kids get started doing that?  There are so many opportunities for technology professionals.  We are at the beginning of this code movement – but we are lacking in teachers that can teach these skills.  Look for opportunities to give back in the school district where your kids are.  We need more than after-school programs and camps.  Black Girls CODE has over 2000 volunteers across the US, there are multiple chapters.  There is a need to talk to students and parents about what we (as women) do in our careers.  (ES: I find it interesting that she mentioned that parents need to hear that discussion as well.)

Kimberly believes that kids can start learning about technology at grade 1.  (ES: I agree – my kids have had a tech class since kindergarten.  Last year, as a 3rd grader, my son put together a PowerPoint presentation.)  Starting to introduce technology in high school is too late.  We need computer science to be counted as a high school credit – it shouldn’t take the place of math or science, it’s in addition.

Denise: Many companies have released diversity statistics.  Does Kimberly talk to attendees about the culture of tech and what it might be like to have a career in tech.  Kimberly states that they do – they try to prepare students to be active participants, and also prepare them for what challenges they might face within the data environment.  Changing the community is not quick thing – it’s a continual effort and requires some difficult conversations (then followed by action).

Over half of the women who enter tech fields drop out at the half way point.  Kimberly says she can relate to this personally.  She understands what it’s like to get in to the career and then see the glass ceiling.  Often, women don’t have the support network to break through that glass ceiling.  The role of mentors, sponsors, and advocates is so important.  On average, most women CEOs come to one company and stay there for 20+ years – that’s how the majority of women CEOs get there.  Women need to stay in the pipeline longer in order to get to the top – but a welcoming and positive environment will help.  Advocates and sponsors for women, within those communities, are needed.  Need more male advocates and mentors to help women get to the next level.  Also, women need to be willing to take the risk to get to that next level.

Denise opens up the discussion for questions from the audience.  It’s mentioned that only 15% of attendees here at PASS Summit are women.  (ES: Really?  15%?)

One of themes from today and Kimberly’s message: mentors are needed.  Both and male and female.  Kids need role models, college students need them, women in technology need them.  (ES: I’d argue that everyone needs a mentor.  Ask yourself: can you be a mentor to someone?  I bet you can.  And don’t be afraid to go ask for – seek out – a mentor for yourself.)

Work culture cited as a top reason that women leave technology.  How do we change that?  Kimberly says to hire more women.  If there’s a company with its heart in diversity, and there’s isolation in the company still, need to change it from the ground up and from the top down, and to do that, need to get more women into the organization.  (ES: That’s not a complete answer, in my opinion, I think it’s more than just getting more women into a company.  You have to understand what the barrier is – what’s the resistance?  Then, you need to figure out how to change that.  And I don’t know if it’s a one-size-fits-all in terms of the barrier – there might be a huge variety of barriers.)

Input from an attendee: go to local school career fairs and talk about IT.  The issue isn’t having to choose between two candidates, it’s trying to get one qualified candidate.

Jes asks how we can get kids to understand that technology skills are important – they’re not just a degree.  Kimberly – we agree, technology skills provide just one tool in a person’s toolkit.  This is why it’s important to get computer science into school, so then it becomes a tool that they can use as they’re learning science, math, and even in non-science courses.

As women we need to be advocates for each other.  (ES: Agreed, we do.)

Kalen has a challenge to parents: talk to your boys about smart women and how they’re not someone to fear.

One of my mentors, Allen White, stands up to ask a question.  Allen has been in IT for 40 years.  He asks, “What can I focus on so I don’t make “bad” choices, since I am not a female, nor a person of color?”  Kimberly tells him to be cognizant, make his company inclusive, to help someone who’s “different” from him.  He’s done all that :)

PASS Summit 2014: Day 2

We’re kicking off Thursday’s PASS Summit keynote in about 5 minutes, and the good news is that I have network connectivity today and I’ll really be live-blogging today.  Stay tuned for updates throughout the morning!

Ok, I guess I need to start with this picture of Brent Ozar and Grant Fritchey:

Brent and Grant with their rainbow leggings to support Doctors Without Borders

Brent and Grant with their rainbow leggings to support Doctors Without Borders

Brent and Grant will be wearing these lovely leggings today when they present as part of an effort by to help raise money for Doctors Without Borders.  You can still donate!

8:15 AM

We’re off and running with Adam Jorgensen, PASS EVP of Finance.  Adam is going to provide an update about the financial status of PASS.  Funny enough, doing this at Summit satisfies the requirements of the by-laws.  The largest source of revenue is the PASS Summit (not a surprise): 96% of the revenue for PASS is generated by Summit and the Business Analytics conference.  The money raised goes to provide activities throughout the community, and there are reserves of over one million dollars (pretty good).  These funds protect the PASS Community in case the Summit is cancelled due to a natural disaster.  In fiscal year 2016, PASS wants to focus on projects already scoped and also provide funds for new projects that are yet to be determined.  PASS publishes the budget every year.  The community members have access to this, and starting in 2016 portfolio-level budgets will be published so it’s easier to drill into the areas of PASS which interest members the most.  The focus in 2015 includes conferences, the global alliance program, investing in IT, community events, data culture, and the business and data analytics community.

8:22 AM

Adam finishes up and PASS President Tom LaRock comes on stage.  Tom takes a few minutes to say goodbye to members who are ending their term on the board, including Sri Sridharan and Olivier Matrat.  Sri managed the Volunteer profile within PASS and did a phenomenal job trying to bring more volunteers into the community to help PASS.  Tom also introduces new Board members: Sanja Mishra and Grant Fritchey.  Next up is Denise McInerney, EVP of Marketing.

Denise mentions that over 5000 people are watching today’s keynote online on PASStv in over 113 countries.  Denise starts by talking about her involvement with PASS, which started back in 2002 with a session she attended by Kimberly Tripp. (ES: SQLskills shout out!)  Denise then got involved locally and at the national level.  When you volunteer for PASS you help other members, and you broaden your own network.  A point from Denise: many of the people she met in the beginning are the ones she still turns to.  Denise announces this year’s PASSion Award winner: Andrey Korshikov, who is based in Russia and a SQL Server MVP and BI Developer.  Andrey is a PASS Regional Mentor and the founder of the Russian VC.  He’s managed four SQLSaturdays and three Russian editions of 24 Hours of PASS.

Denise also mentions those who were also nominated for the PASSion award – but I couldn’t type them all fast enough :)  She then highlights the PASS Outstanding Volunteers that have been recognized through the year and asks them to stand and be recognized.  (ES: It takes a village – there are so many fantastic people who contribute to this community.) 

On Friday, from 2:15 to 2:45, in room 307/308, there will be a Business Analytics Direction Board Discussion.  If you want to provide feedback about the Business Analytics Conference (taking place April 20-22, 2015 in Santa Clara, CA) and/or this direction that PASS is going, please attend the discussion.  Denise also reminds people to update their PASS profile, particularly if you want to volunteer and and get involved.

The next PASS Summit will be in Seattle, October 27-30, 2015.  Registration is already open!

 8:35 AM

Dr. Rimma Nehme finally takes the stage for her keynote: Cloud Databases 101.  She is a Principal Research Engineer at the Microsoft Jim Gray Systems Lab.

Dr. Nehme has been watching this conference for the past 5 years, and starts by thanking the organizers for inviting her, and mentions Dr. DeWitt.  And she was thinking about how she could be like Dr. DeWitt, and then realized, “trying to be a man is a waste of a woman”.  She won’t try to be like Dr. DeWitt, she will just be herself.  Yes.  Dr. Nehme was born in Belarus, she knows a little bit about databases from an academic and real-world perspective, and she is learning a lit bit about business too.  (Dr. Nehme is getting her MBA in her “spare time”…seriously…and did I mention that she’s also a mom of two kids?  SO impressive.)  Dr. Nehme is a big fan of the PASS Community.

Today’s topic is: What is a cloud database?  Our roadmap for today:

  • Why Cloud?
  • What’s a Cloud Database?
  • How are they built?
  • What’s my role as a DBA?
  • Summary

Cloud technology is still relatively new, and it has “Shiny Object Syndrome” around it.  Dr. Nehme’s goal is to explain why cloud is special.  Basic equation to remember is that cloud = service.  More precisely defined: the cloud is computing and software resources that are delivered on demand, as a service that is always on, accessible from anywhere, and at any time.  This is also known as the 5the utility.  Why is it called cloud computing?  Blame the network people (not database people).  Cloud computing characteristics:

  • on-demand self-service – demand for resources can be filled automatically
  • location transparent resource pooling – resources are pooled to several customers
  • ubiquitous network access – all resources available over the network that allows data exchange
  • rapid elasticity – capability provided on-demand when needed, then releases
  • measure service with pay per use – resource charges as based on the quantity used

Think about it: one woman or a man, and a credit card, can tap into some of the largest computing solution in the world.

A brief history: the wave of computing started in the 1960s.  The concept of computation was born in this time by one of the MIT professors.  In the 1990s, the first cloud application was offered.  In the 2002, Amazon Web Services was launched, and Windows and Google launched offerings in the 2008 timeframe.

Question: Where does the cloud live?  In a data center.  Let’s go on a virtual tour of a Microsoft Data Center.  The data center in Chicago looks like a fancy trailer park.  What’s inside those big containers?  Lots and lots of servers.  When we think of a data center we think of lots of servers, raised floors, etc.  There is more to it, there are transformers, cooling towers, chillers, UPS’, powers, and people.  One way to describe a data center is by its efficiency.  Optimizing for energy efficiency is a good thing.  We are socially responsibility to pay attention to our use of resources.  One way that efficiency is calculated is by using PUE = power usage effectiveness.  The formula is the total facility power divided by IT equipment power.  This is valuable as a broad efficiency ratio.  The PUE ration for a modular data center (hosting cloud resources) is 1.15, whereas for a traditional data center it is 2.0.  Interestingly enough, the cooling for the modular data center is 0%.  One example of how this is done is swamp cooling (aka evaporative cooling)…put cold water in front of fans.  ES: Rob Farley tells me this is how its done in Australia.  Data centers have significantly evolved since the late 1980s.  There are over 100 data centers in more than 40 countries – more than 1 million servers.  What does Microsoft consider with site selection?  There are over 35 factors, the top 3 are proximity to the customers, energy and fiber infrastructure, and the skilled workforce.

The main takeaways for why cloud: elasticity, no cap ex, pay per use, focus on business, and fast time to market.  This is why cloud computing is special.

What is a cloud database?  Everything in the cloud is a service.  So you’re getting a database, as a service.  Cloud services has 3 layers: infrastructure, platforms, and applications (and these are also all services).  The Microsoft Cloud has the same thing – infrastructure services, platform services (e.g. Windows Azure, SQL Azure), application services.  When you have a data center on site, you manage everything.  When it’s infrastructure as a service, part of that stack is outsourced to a vendor.  With platform as a services, you’re responsible for the application and data – everything else is outsourced to the vendor.  With software as a service – you outsource everything.  Dr. Nehme takes this and then does “pizza-as-a-service” analogy:

  • On Premise = you buy everything and make the pizza at home
  • IaaS = take and bake (pick up the pizza, you cook it at home)
  • PaaS = pizza delivered
  • SaaS = dining in the restaurant

IAAS in the database world – you must still manage provisioning, backups, security, scaling, failover, replication, tuning, performance, etc.  Dr. Nehme calls this lift and shift – take an earth version of a database and put it in the cloud.  Existing applications don’t need to be modified.  Just need to point to version of the DBMS in the cloud.

PAAS = DBMS as a service – select the cloud vendor, select a DBMS.  Here, the cloud vendor manages provisioning, backups, security, tuning, failover, etc.  There might be some changes to the language surface compared to an earth version of a database.

SAAS = select the cloud vendor, select a cloud app (SharePoint).  The whole stack is outsource to the cloud vendor.

Database as a service examples:

  • Managed RDBMs (SQL Server)
  • Managed No SQL (Doc DB, MongoHQ)
  • Cloud-Only DBaaSS (Dynammo DB, Google F1)
  • Analytics-as-a-Service (HDInsight, EMR)
  • Object Stores (Azure storage, S3)

Why virtualization?  It’s a huge enabler for cloud computing.  Unfortunately, many servers are grossly underutilized.  Virtualization developed to put resources back to work.  However, there are bottlenecks with these resources.  What can be virtualized?  CPU, network, memory, and disk.  Keep in mind that there is no free lunch.  Virtualization comes with limitations.  Lose direct access to the computing resources.  Now have an indirect path.  Also, hiding the details of physical resources is unfortunate in terms of configurations.  In addition, virtualization always causes some degree of performance penalty.  Use cases are consolidation, migration and load balancing, and high availability.    For consolidation, if CPU requirements are high for one server, and IO requirements are high for a second server, consolidating those two might be ideal (and can also equate to energy-savings).

With migration and load balancing, assume one machine with a VM with a RDBMS that gets overloaded.  It could be migrated to another machine to help maintain performance.  And with high availability – one machine with a VM and then backup machine with VM image, will detect a failover, restart the image so the server stays up and available.

There are four common approaches to multi-tenancy (with a lodging analogy):

  • Private OS (SQL Server in a VM) – private apartment
  • Private Process/DB (MongoHQ) – private room
  • Private Schema (Azure SQL DB) – share room
  • Shared Schema (SalesForce) – share bed

What’s the big deal with this?  When you consider database as a service, what are the requirements for your database, for your data?  If application independence is important, don’t go with a shared schema approach.  You must do cost-benefit analysis.  Given pros and cons, what works best for you?

Service Level Agreements…when people talk about the cloud they talk about SLAs.  It’s a common understanding about services, guarantees, and responsibilities.  There is a legal component and a technical component.  Service level objectives are measurable characteristics such as performance goals, deadlines, constraints, etc.  Think of this in terms of availability and “nines”.  If you require four nines (99.99%) up time, that’s about 4 minutes of downtime per month.  Three nines (99.9%) is about 43 minutes per month.  Just one nine can make a big difference.  Container based hardware is three nines-reliable, but with SQL DB they are delivering four nines-reliability.

Three main concepts behind Azure SQL DB:

  • Account – 0 or more servers
  • Server – 1 or more databases
  • Database – standard SQL objects

This was designed with high availability in mind.  This means that there are multiple replicas of data.  There is a primary and two secondaries.  If a node goes down, the secondary becomes a primary, then replicate again so end up with two secondaries.  Reads are completed on primary, writes replicated to secondaries.  Four layers:

  • client – used by application to communicate directly to SQL Database
  • services – the gateway between the clients connecting to the SQL DB and the platform layer where computation occurs; provisioning, billing, routing for connections
  • platform – physical services that support the services layer above, includes SQL Server, management services
  • infrastructure – IT admin of physical hardware and OS

Applications connect to the internet, go to the Azure cloud, get to the load balances, hit the gateway which are connected to the SQL DB nodes, and then under all that is the scalability and availability fabric which does failover, replication and load balancing.  What does the SQL node look like?  It’s a machine with a SQL instance with a single physical database for the entire node.  The database files and logs are shared across every logical database – might be sharing log files with someone else.  Each logical database is a silo with its own independent schema (sharing a room analogy).

What we if create a database or run a query, how does it work?  The Azure service will identify where to put the primary database (when creating a new one), then put secondaries on two other machines.  When a user comes in to do a query, the SQL Azure gateway service will identify where the primary is located, get to it, perform computation, and return results to user.  If you want to know more – go to the sessions here at PASS :)

Next up: my role as a DBA.  From Dr. Nehme: “I have to be honest, I tried to put myself in your shoes.”  She asks, do we still need a DBA in the cloud era?  Dr. Nehme says yes.  Cloud doesn’t have to an either/or choice.  You can augment on-premise systems with cloud (remember the stretched tables example for yesterday).  This is the time to refresh your skills and adjust to this era in the cloud.  The cloud was not designed to be a threat to DBAs.  The number of DBAs vs. the number of database apps vs hardware computer capacity.  The number of DBAs is much smaller than both.  This is where cloud computing can help.  Address issue of underutilized hardware and alleviate some of the work of overburdened DBAs.  Dr. Nehme’s recommendation is to take current skills, add cloud skills, and call yourself a Cloud DBA.

Some key things to remember:

  1. cloud database = a service, designe dto reduce admin and operational costs (pay as you go, elasticity), there is a wide spectrum of solution (rent a database, cloud database).
  2. If you get confused about cloud deployment options, remember the pizza analogy
  3. Do the cost benefit analysis, and you need to embrace the cloud.  It presents a lot of opportunities.

Dr. Nehme finishes up and takes a minute to thank Dr. DeWitt and has him come on stage. She hints at possibility a keynote with both of them in the future.  I’d love that, but I’d also be happy to just hear Dr. Nehme again :)  Great session.  Perry is overwhelmed…

Perry after Dr. Nehme's talk...

Perry after Dr. Nehme’s talk…


Edit: 10:09AM In my original publishing I referred to Dr. Nehme as Rimma…and I think it’s because Dr. DeWitt always refers to her that way (and she refers to him as David).  I updated the post to fix that.  And also wanted to add a new pic (Dr. Nehme’s keynote and having the chance to chat with her was one of this week’s highlights):

Dr. Nehme and me

Dr. Nehme and me

PASS Summit 2014: Day 1

It’s Day 1 of the PASS Summit, I’m live-blogging the keynote, and I can’t get on the internet. My DR strategy (hotspot on my phone) is failing as well. This may be late getting posted, but that’s ok. The show must go on.

Perry and I, ready for the keynotePerry and I, ready for the keynotePerry and I, ready for the keynote

Perry and I, ready for the keynote

This morning we’ll hear from the Microsoft team, including:

  • T.K. “Ranja” Rengarajan, Corporate VP of the Data Platform, Cloud & Enterprise at Microsoft
  • James Phillips, General Manager of Power BI at Microsoft
  • Joseph Sirosh, Corporate VP of Information Management and Machine Learning at Microsoft

While setting up at the blogger’s table this morning, PASS EVP of Finance and Governance Adam Jorgensen introduced me to Brendan Johnston, who joined the PASS team five weeks ago and is going to work on marketing for PASS. I had a couple minutes to hear where he came from (Sony) and what he’s been working on so far (getting better messaging out to the community, including some additional communication in the weeks leading up to the Summit). He’s been busy already! I’m interested to see how PASS works to bring in data professionals who still do not that PASS exists. It’s a challenge to bring a group that doesn’t know you exist into the fold.

8:21 AM

And we’re off, with PASS President Tom LaRock kicking off the day.

This is the 16th annual Summit. As a reminder, PASS TV is streaming the keynote today and tomorrow! Also, I’ll be on PASS TV today (Wednesday) at 2:50PM PST :) Tom points out that over 50 countries are represented here at the Summit from thousands of companies, including first-timers, veteran members, leaders, volunteers, and Microsoft employees.

“This is our community.”

“The people who are next to you will help shape your career, and you shape yours.” ES: I have Allen White on one side of me, Glenn Berry on the other.  Yes, these two have shaped my career.

Tom introduces the PASS Board and asks attendees to share thoughts, comments, questions, and concerns with them throughout the week. On Friday at 1:15PM the Board will have an open Q&A in room 307 and 308.

The Summit started in 1999. Microsoft and CA Technologies had a vision of a community that would focus on Microsoft technology. With the content and networking from that first Summit, the community began to grow. Today, the PASS community reaches over 100,000 data professionals in over 100 countries with 285 chapters world-wide. PASS has provided over 1.3 million training hours since its inception.

Tom asks where you will be in 15 years. ES: Allen says to me: Retired. Ha, he’s a funny guy :) When we grow our skill set, we grow our opportunities.

“Growth is never by mere chance; it is the result of forces working together.” –James Cash Penney

Tom states that this quote represents PASS, as PASS has become a cornerstone in our careers. We say to others “come with me, and check this out.”  Tom challenges us to get involved and grow. The best connections you can make are only a handshake away, right here, this week. Talk to someone. Connect, share, learn. Tom reminds people to not let growth end after Summit. Stay engaged throughout the year.

This year PASS has amazing opportunities for attendees –this includes 200 training sessions and instructor-led workshops, the chance to get certified, the SQLCat team, Women in Technology, Community Zone, and more.  Tom also takes a moment to mention the partners for the Summit. Without the sponsors, this would not possible. ES: Please, PLEASE, take the time to visit the sponsors this week and thank them for all that they do.  And I’ll give a shout out to one sponsor, SQL Sentry, right here for all they do for the community, including this morning’s #SQLRun which had over 100 people.  Nicely done Jes and SQL Sentry.

8:40 AM

The Microsoft team takes the stage, with Ranga up first. I had a chance to hear this team earlier in the week, and I was very impressed, especially with James Phillips. I probably shouldn’t have a bias, but his passion and past experience will serve this community well, I believe.

Ranga starts with his background – born in India, came to the US, received his education under Dr. DeWitt at Wisconsin (ES: one of my personal favorites – Dr. DeWitt, not Wisconsin) and then went to Silicon Valley and has been there since. Ranga’s family includes two daughters that he is encouraging to get into tech.  Ranga tells a story about how he loves maps (so do I!) – I find this funny for a man since they never ask for directions (ohhhh, so sorry!). Ranga loved MapQuest, and then GPS (ES: though his wife doesn’t like the GPS Jane’s voice…I don’t either, have you ever heard her pronounce Spanish street names? Hilarious).

There is an incredible number of devices proliferating right now. These devices are growing at astronomical rates. With these devices comes a lot of data generated and consumed, which is projected to grow – 41% every year. EVERY YEAR?!?!?! How will we handle this? Projecting 1.6 trillion dollars with this data based on trending. Can see personal instances where you can see how data is changing how you work and live. This a HUGE opportunity for us. At Microsoft, data is the thing that will light up future productivity. When we talk about productivity, we think it’s coming in the form of different types of data, and people wanting that data. NOW. The opportunity and challenge is take that data and make a difference for everyone. People use that data to make decisions in their life. Microsoft creates the platform to provide the insight to make those decisions.

This data culture will allow everyone to do more and achieve more in their decisions. It’s a cycle – if you capture data and manage it, and get insights from it, and you visualize and make decisions…it creates the need for more data. Microsoft experiences this now. This data platform is divided into granular areas with multiple capabilities. Ranga wants to talk about capturing and managing it. Must combine data across multiple areas (cloud, on-prem) and this platform must be comprehensive. Success from Microsoft is a solution that doesn’t require compromises, it is a culture of AND (not OR), and you CAN have it all. Microsoft can do in-memory and on disk, optimized for the hybrid environment (don’t take sides for on-prem and cloud, do you what’s right for your business!), structured and unstructured data, scale up or scale out. No limit on what you can do with this data.

Key characteristics necessary to do more and achieve more:

  • Capture diverse data
  • Achieve elastic scale
  • Maximize performance and availability
  • Simplify with cloud

Ranga talks about the technologies that can help with this:

  • Azure Document DB (NoSQL DB service, schema-free, ACID to eventual consistency)
  • Azure HDInsight (Apache Hadoop service with HBase and Storm)
  • Analytics Platform System (Polybase (SQL and Hadoop) appliance
  • Azure Search (Fully managed search service for mobile apps

For scaling, need to scale up and scale out. Microsoft has:

  • SQL Server 2014 with Windows Server 2012R2 (scale up to 640 logical cores, 64 vCPUs per VM, 1TB of memory per VM)
  • SQL Server in Azure VMs (new massive G-services VMs – base on the market)
  • Azure SQL Database (taking scale out approach – hyper-scale across thousands of DBs)

“Use the best tool for the job.”  ES: Yes. I ask “What problem are you trying to solve?”

Tracy Daugherty from Microsoft takes the change for a demo – I met him the other night, good guy. He’s talking about how to find inventory he wants to move (orange pumpkins…Halloween is over, time to get that product out the store and make way for holiday decorations!). Tracy is using Azure DB for the inventory, and showing JSON code that gets uploaded to update customer facing pages. When capacity increases, need to be able to support that – and can be done via elastic scale. Tracy talks about sharding which can be time-based, based on size, etc. It’s effectively one database, but broken out and spread across multiple shards.

This example is one of combining multiple products into one solution – get the right tools. We are at the beginning of the amazing possibilities here. All of these services are available in preview right now. Tracy notes: a new feature was GA’d last week (made generally available). Tracy wants to take main database and make sure it’s replicated across regions via geo-replication. Tracy picks a server over in Asia, and it replicates the existing database across the world in three easy steps.

Ranga says Microsoft provide the best up-time for any solution – four nines (99.99%). SQL DB is on a tear. The same engine is used for both SQL DB and SQL Server. There are a million databases running in SQL DB, and Microsoft is now truly understanding what data professionals go through. All fixes deployed to SQL DB get deployed to the next box version of SQL Server. SQL Server 2014 is getting great reception, the in-memory OLTP is incredible – no one else is able to do that. One engine can handle multiple workloads. Microsoft is taking advantages of all the things happening in Azure. On-prem you can connect to the cloud in a trusted manner which will allow you to extend your solution naturally.

Think about the world of differently. We have looked at data as carefully orchestrated. The new world says, take the data, put it in the right engine, and leave it in the cloud. Know that you can get insight from that data at any time. Azure is now becoming the new data layer. Ranga mentions Stack Overflow (ES: I see Brent do a fist pump..but then some frustration as Stack’s solution is misrepresented) and all that they are able to do with their commodity hardware and SSDs – it scales out well. They use the software in very clever ways. This is awesome to see. Ranga also mentions Samsung who has seen 24x improvement in performance with in-memory OLTP.

Ranga announces a major update to Azure SQL DB later this year that will be in preview. It represents an incredible milestone for Microsoft. This includes:

  • Leap in TSQL compatibility
  • Larger index handling
  • Parallel queries
  • Extended events (ES: YES!)
  • In-memory ColumnStore for data marts

More capabilities will roll out across multiple environments.

9:18 AM

Ranga brings Mike Zwilling, one of the Microsoft engineers, up on stage. A little background – holiday season, expecting an increase in transactions. People also want more real time insight. What if you could run analytics directly on OLTP data? (ES: Funny enough, I know companies that do that right now.) Mike gives a URL that viewers can go to and “buy” something – this is going to generate workload. Mike then shows the performance live. He shows the live view on the OLTP data, and shows performance via PerfMon.  (ES: I still love PerfMon.) Mike points out that the supporting table is using in-memory OLTP and nonclustered columnstore.

Mike talks about new functionality coming – the ability to stretch a table into Azure. (ES: I find this EXTREMELY exciting, I have a customer that benefit from this right now.) History tables, for example, can be stretched to exist the local server as well as into Azure. The older data is moved, behind the scenes, to Azure.  (ES: Awesome. The only thing I don’t love? There’s a DMV to look at rows moved to Azure named db_db_stretch_stats. Dear MS: you need to take stats out of that name. Please.) Mike demos how, in the event of a failover, you can restore the local database, and when it finished it is synchronized with Azure to bring it to the same point in time as what’s on-prem. Pretty cool. Ranga explains that the stretch concept is a logical – have an on-prem database that you extend, and it occurs in an invisible way.

9:28 AM

Joseph Sirosh takes the stage, he spent about 9 years at Amazon before joining Microsoft about a year ago. PASS is an amazing community. Communities come together to learn. Joseph wants to do the wave. (ES: Huh. Didn’t see that one coming. We’re going to do “PASS community rocks” as wave. We’ll do this if you want Joseph, three times. I’m a cheerleader and I’m not loving it. Where are everyone’s spirit fingers? Golf clap from Allen.)

On to machine learning – this is something Joseph is EXTREMELY passionate about. He mentions Storm. (ES: Did I ever tell you that my husband wanted to name our son Storm? That was voted down immediately.) Azure machine learning is about learning patterns in the past and anticipating what will happen in the future. Joseph brings Sanjay Soni to the stage. He asks how many people love Christmas shopping. I don’t. It stresses me out. More with Pier 1 and something about last minute shopping (that sounds familiar). Sanjay wants to figure out what items to put on end-caps. He’s going to talk about using Kinect sensors. Gather data about where shoppers spend their time. What products they are lingering around. This is cool, but as a consumer, I don’t like it. The heat map was created using PowerMap in Excel. Looking at last three days of data – behind the scenes. Azure data factory – something new in the Azure portal. SSIS to the power of X in the cloud – all kinds of data sources coming in.   Browser-based authoring environment. 1500 lines of code with a third-party app to do the analysis that 100 lines of JSON code does. JSON is a definite buzz technology this week. I do appreciate Sanjay’s enthusiasm. Rugs and furniture was the hot data to put on the aisles. (ES: SERIOUSLY? Steve Jones tells me to lighten up. This from a man wearing a red hate with stripes on it. Maybe he needs to be more serious?!? :) Ok, so real-time the hot items are candles (?) and bar & wine. I told you…not rugs and furniture. Wine is NOT surprising. The product is real-time dashboard. It’s the information coming from the sensors, data going into an Azure database. With only 14 lines of code can do stream analytics (versus 1000 from other vendors). Streaming is incredibly powerful.

“Let’s predict the future so we can change the future.”

Azure machine learning – look at loyalty customer information. Predict what a customer will buy based on their last purchase. (ES: Hm. I’m going to start messing with the data and buy random stuff.  (From where is this crankiness coming from?!?!) Are you getting junk emails from retailers that accurately predict what you’re going to buy? Is it wine? Ok.)  Last demo from Sanjay, on his phone, he’s a member of the loyalty program. The app welcomes him, and gives him a list of products he might be interested in, based on previous purchases. This includes beer mugs. The app called into machine learning. Fascinating. And really scary.

9:49 AM

James finally takes the stage. He is new to Microsoft, here for just over two years. Came from the Valley and spent that time building two companies. He is running the data experiences team. Running the Power teams and analytics. How do we bring data to people?  (ES: ok, I admit that I’m fading)

James mentions that he was watching the Twitter feed backstage. That’s risky, and impressive. Do you change the angle of your talk?

One thing that makes Microsoft different: the ability to tie back from the clout to on-prem. Microsoft is looking to build a data culture using PowerBI multiple capabilities. James moves into a demo, which he does himself. Props for that. Sticking with Pier1 for the demo (he kind of has to, but recognizes the feedback that’s been given on Twitter). James wants to get a pulse on the business – does this using a PowerBI dashboard – it’s a diagnostics component. James wants to understand why there’s a trend with candle purchases. Me too. James searches for a variety of pieces of information he wants to see, pins that information to his dashboard, and then arranges it in a way that the wants to see. I admit, the flexibility of the dashboard is pretty slick. The challenge is understanding how to get there.

(ES: Seriously, I love that James is doing his own demo.)

10:02 AM

James finishes up, and Ranga comes back. He thanks Pier1 for their help with the demos. One more thing from Ranga…the Azure Machine learning is available for free on a trial basis today. Ranga wants us to use that today. Again, he states that Microsoft is building an amazing data platform. Think of what we’re seeing today as a comprehensive platform.

“Be the hero of your data-driven business. Think through the one thing that captured your imagination, and then go connect and learn with that. This is the time for data. The world is excited about data. You are the guardians of data. Together we can change the world. We can do more together!”