It’s Day 1 of the PASS Summit, I’m live-blogging the keynote, and I can’t get on the internet. My DR strategy (hotspot on my phone) is failing as well. This may be late getting posted, but that’s ok. The show must go on.
This morning we’ll hear from the Microsoft team, including:
- T.K. “Ranja” Rengarajan, Corporate VP of the Data Platform, Cloud & Enterprise at Microsoft
- James Phillips, General Manager of Power BI at Microsoft
- Joseph Sirosh, Corporate VP of Information Management and Machine Learning at Microsoft
While setting up at the blogger’s table this morning, PASS EVP of Finance and Governance Adam Jorgensen introduced me to Brendan Johnston, who joined the PASS team five weeks ago and is going to work on marketing for PASS. I had a couple minutes to hear where he came from (Sony) and what he’s been working on so far (getting better messaging out to the community, including some additional communication in the weeks leading up to the Summit). He’s been busy already! I’m interested to see how PASS works to bring in data professionals who still do not that PASS exists. It’s a challenge to bring a group that doesn’t know you exist into the fold.
And we’re off, with PASS President Tom LaRock kicking off the day.
This is the 16th annual Summit. As a reminder, PASS TV is streaming the keynote today and tomorrow! Also, I’ll be on PASS TV today (Wednesday) at 2:50PM PST 🙂 Tom points out that over 50 countries are represented here at the Summit from thousands of companies, including first-timers, veteran members, leaders, volunteers, and Microsoft employees.
“This is our community.”
“The people who are next to you will help shape your career, and you shape yours.” ES: I have Allen White on one side of me, Glenn Berry on the other. Yes, these two have shaped my career.
Tom introduces the PASS Board and asks attendees to share thoughts, comments, questions, and concerns with them throughout the week. On Friday at 1:15PM the Board will have an open Q&A in room 307 and 308.
The Summit started in 1999. Microsoft and CA Technologies had a vision of a community that would focus on Microsoft technology. With the content and networking from that first Summit, the community began to grow. Today, the PASS community reaches over 100,000 data professionals in over 100 countries with 285 chapters world-wide. PASS has provided over 1.3 million training hours since its inception.
Tom asks where you will be in 15 years. ES: Allen says to me: Retired. Ha, he’s a funny guy 🙂 When we grow our skill set, we grow our opportunities.
“Growth is never by mere chance; it is the result of forces working together.” –James Cash Penney
Tom states that this quote represents PASS, as PASS has become a cornerstone in our careers. We say to others “come with me, and check this out.” Tom challenges us to get involved and grow. The best connections you can make are only a handshake away, right here, this week. Talk to someone. Connect, share, learn. Tom reminds people to not let growth end after Summit. Stay engaged throughout the year.
This year PASS has amazing opportunities for attendees –this includes 200 training sessions and instructor-led workshops, the chance to get certified, the SQLCat team, Women in Technology, Community Zone, and more. Tom also takes a moment to mention the partners for the Summit. Without the sponsors, this would not possible. ES: Please, PLEASE, take the time to visit the sponsors this week and thank them for all that they do. And I’ll give a shout out to one sponsor, SQL Sentry, right here for all they do for the community, including this morning’s #SQLRun which had over 100 people. Nicely done Jes and SQL Sentry.
The Microsoft team takes the stage, with Ranga up first. I had a chance to hear this team earlier in the week, and I was very impressed, especially with James Phillips. I probably shouldn’t have a bias, but his passion and past experience will serve this community well, I believe.
Ranga starts with his background – born in India, came to the US, received his education under Dr. DeWitt at Wisconsin (ES: one of my personal favorites – Dr. DeWitt, not Wisconsin) and then went to Silicon Valley and has been there since. Ranga’s family includes two daughters that he is encouraging to get into tech. Ranga tells a story about how he loves maps (so do I!) – I find this funny for a man since they never ask for directions (ohhhh, so sorry!). Ranga loved MapQuest, and then GPS (ES: though his wife doesn’t like the GPS Jane’s voice…I don’t either, have you ever heard her pronounce Spanish street names? Hilarious).
There is an incredible number of devices proliferating right now. These devices are growing at astronomical rates. With these devices comes a lot of data generated and consumed, which is projected to grow – 41% every year. EVERY YEAR?!?!?! How will we handle this? Projecting 1.6 trillion dollars with this data based on trending. Can see personal instances where you can see how data is changing how you work and live. This a HUGE opportunity for us. At Microsoft, data is the thing that will light up future productivity. When we talk about productivity, we think it’s coming in the form of different types of data, and people wanting that data. NOW. The opportunity and challenge is take that data and make a difference for everyone. People use that data to make decisions in their life. Microsoft creates the platform to provide the insight to make those decisions.
This data culture will allow everyone to do more and achieve more in their decisions. It’s a cycle – if you capture data and manage it, and get insights from it, and you visualize and make decisions…it creates the need for more data. Microsoft experiences this now. This data platform is divided into granular areas with multiple capabilities. Ranga wants to talk about capturing and managing it. Must combine data across multiple areas (cloud, on-prem) and this platform must be comprehensive. Success from Microsoft is a solution that doesn’t require compromises, it is a culture of AND (not OR), and you CAN have it all. Microsoft can do in-memory and on disk, optimized for the hybrid environment (don’t take sides for on-prem and cloud, do you what’s right for your business!), structured and unstructured data, scale up or scale out. No limit on what you can do with this data.
Key characteristics necessary to do more and achieve more:
- Capture diverse data
- Achieve elastic scale
- Maximize performance and availability
- Simplify with cloud
Ranga talks about the technologies that can help with this:
- Azure Document DB (NoSQL DB service, schema-free, ACID to eventual consistency)
- Azure HDInsight (Apache Hadoop service with HBase and Storm)
- Analytics Platform System (Polybase (SQL and Hadoop) appliance
- Azure Search (Fully managed search service for mobile apps
For scaling, need to scale up and scale out. Microsoft has:
- SQL Server 2014 with Windows Server 2012R2 (scale up to 640 logical cores, 64 vCPUs per VM, 1TB of memory per VM)
- SQL Server in Azure VMs (new massive G-services VMs – base on the market)
- Azure SQL Database (taking scale out approach – hyper-scale across thousands of DBs)
“Use the best tool for the job.” ES: Yes. I ask “What problem are you trying to solve?”
Tracy Daugherty from Microsoft takes the change for a demo – I met him the other night, good guy. He’s talking about how to find inventory he wants to move (orange pumpkins…Halloween is over, time to get that product out the store and make way for holiday decorations!). Tracy is using Azure DB for the inventory, and showing JSON code that gets uploaded to update customer facing pages. When capacity increases, need to be able to support that – and can be done via elastic scale. Tracy talks about sharding which can be time-based, based on size, etc. It’s effectively one database, but broken out and spread across multiple shards.
This example is one of combining multiple products into one solution – get the right tools. We are at the beginning of the amazing possibilities here. All of these services are available in preview right now. Tracy notes: a new feature was GA’d last week (made generally available). Tracy wants to take main database and make sure it’s replicated across regions via geo-replication. Tracy picks a server over in Asia, and it replicates the existing database across the world in three easy steps.
Ranga says Microsoft provide the best up-time for any solution – four nines (99.99%). SQL DB is on a tear. The same engine is used for both SQL DB and SQL Server. There are a million databases running in SQL DB, and Microsoft is now truly understanding what data professionals go through. All fixes deployed to SQL DB get deployed to the next box version of SQL Server. SQL Server 2014 is getting great reception, the in-memory OLTP is incredible – no one else is able to do that. One engine can handle multiple workloads. Microsoft is taking advantages of all the things happening in Azure. On-prem you can connect to the cloud in a trusted manner which will allow you to extend your solution naturally.
Think about the world of differently. We have looked at data as carefully orchestrated. The new world says, take the data, put it in the right engine, and leave it in the cloud. Know that you can get insight from that data at any time. Azure is now becoming the new data layer. Ranga mentions Stack Overflow (ES: I see Brent do a fist pump..but then some frustration as Stack’s solution is misrepresented) and all that they are able to do with their commodity hardware and SSDs – it scales out well. They use the software in very clever ways. This is awesome to see. Ranga also mentions Samsung who has seen 24x improvement in performance with in-memory OLTP.
Ranga announces a major update to Azure SQL DB later this year that will be in preview. It represents an incredible milestone for Microsoft. This includes:
- Leap in TSQL compatibility
- Larger index handling
- Parallel queries
- Extended events (ES: YES!)
- In-memory ColumnStore for data marts
More capabilities will roll out across multiple environments.
Ranga brings Mike Zwilling, one of the Microsoft engineers, up on stage. A little background – holiday season, expecting an increase in transactions. People also want more real time insight. What if you could run analytics directly on OLTP data? (ES: Funny enough, I know companies that do that right now.) Mike gives a URL that viewers can go to and “buy” something – this is going to generate workload. Mike then shows the performance live. He shows the live view on the OLTP data, and shows performance via PerfMon. (ES: I still love PerfMon.) Mike points out that the supporting table is using in-memory OLTP and nonclustered columnstore.
Mike talks about new functionality coming – the ability to stretch a table into Azure. (ES: I find this EXTREMELY exciting, I have a customer that benefit from this right now.) History tables, for example, can be stretched to exist the local server as well as into Azure. The older data is moved, behind the scenes, to Azure. (ES: Awesome. The only thing I don’t love? There’s a DMV to look at rows moved to Azure named db_db_stretch_stats. Dear MS: you need to take stats out of that name. Please.) Mike demos how, in the event of a failover, you can restore the local database, and when it finished it is synchronized with Azure to bring it to the same point in time as what’s on-prem. Pretty cool. Ranga explains that the stretch concept is a logical – have an on-prem database that you extend, and it occurs in an invisible way.
Joseph Sirosh takes the stage, he spent about 9 years at Amazon before joining Microsoft about a year ago. PASS is an amazing community. Communities come together to learn. Joseph wants to do the wave. (ES: Huh. Didn’t see that one coming. We’re going to do “PASS community rocks” as wave. We’ll do this if you want Joseph, three times. I’m a cheerleader and I’m not loving it. Where are everyone’s spirit fingers? Golf clap from Allen.)
On to machine learning – this is something Joseph is EXTREMELY passionate about. He mentions Storm. (ES: Did I ever tell you that my husband wanted to name our son Storm? That was voted down immediately.) Azure machine learning is about learning patterns in the past and anticipating what will happen in the future. Joseph brings Sanjay Soni to the stage. He asks how many people love Christmas shopping. I don’t. It stresses me out. More with Pier 1 and something about last minute shopping (that sounds familiar). Sanjay wants to figure out what items to put on end-caps. He’s going to talk about using Kinect sensors. Gather data about where shoppers spend their time. What products they are lingering around. This is cool, but as a consumer, I don’t like it. The heat map was created using PowerMap in Excel. Looking at last three days of data – behind the scenes. Azure data factory – something new in the Azure portal. SSIS to the power of X in the cloud – all kinds of data sources coming in. Browser-based authoring environment. 1500 lines of code with a third-party app to do the analysis that 100 lines of JSON code does. JSON is a definite buzz technology this week. I do appreciate Sanjay’s enthusiasm. Rugs and furniture was the hot data to put on the aisles. (ES: SERIOUSLY? Steve Jones tells me to lighten up. This from a man wearing a red hate with stripes on it. Maybe he needs to be more serious?!? 🙂 Ok, so real-time the hot items are candles (?) and bar & wine. I told you…not rugs and furniture. Wine is NOT surprising. The product is real-time dashboard. It’s the information coming from the sensors, data going into an Azure database. With only 14 lines of code can do stream analytics (versus 1000 from other vendors). Streaming is incredibly powerful.
“Let’s predict the future so we can change the future.”
Azure machine learning – look at loyalty customer information. Predict what a customer will buy based on their last purchase. (ES: Hm. I’m going to start messing with the data and buy random stuff. (From where is this crankiness coming from?!?!) Are you getting junk emails from retailers that accurately predict what you’re going to buy? Is it wine? Ok.) Last demo from Sanjay, on his phone, he’s a member of the loyalty program. The app welcomes him, and gives him a list of products he might be interested in, based on previous purchases. This includes beer mugs. The app called into machine learning. Fascinating. And really scary.
James finally takes the stage. He is new to Microsoft, here for just over two years. Came from the Valley and spent that time building two companies. He is running the data experiences team. Running the Power teams and analytics. How do we bring data to people? (ES: ok, I admit that I’m fading)
James mentions that he was watching the Twitter feed backstage. That’s risky, and impressive. Do you change the angle of your talk?
One thing that makes Microsoft different: the ability to tie back from the clout to on-prem. Microsoft is looking to build a data culture using PowerBI multiple capabilities. James moves into a demo, which he does himself. Props for that. Sticking with Pier1 for the demo (he kind of has to, but recognizes the feedback that’s been given on Twitter). James wants to get a pulse on the business – does this using a PowerBI dashboard – it’s a diagnostics component. James wants to understand why there’s a trend with candle purchases. Me too. James searches for a variety of pieces of information he wants to see, pins that information to his dashboard, and then arranges it in a way that the wants to see. I admit, the flexibility of the dashboard is pretty slick. The challenge is understanding how to get there.
(ES: Seriously, I love that James is doing his own demo.)
James finishes up, and Ranga comes back. He thanks Pier1 for their help with the demos. One more thing from Ranga…the Azure Machine learning is available for free on a trial basis today. Ranga wants us to use that today. Again, he states that Microsoft is building an amazing data platform. Think of what we’re seeing today as a comprehensive platform.
“Be the hero of your data-driven business. Think through the one thing that captured your imagination, and then go connect and learn with that. This is the time for data. The world is excited about data. You are the guardians of data. Together we can change the world. We can do more together!”