{"id":632,"date":"2009-08-21T11:45:00","date_gmt":"2009-08-21T11:45:00","guid":{"rendered":"\/blogs\/bobb\/post\/Microsoft-Complex-Event-Processing-platform-is-called-StreamInsight.aspx"},"modified":"2013-01-03T23:59:44","modified_gmt":"2013-01-04T07:59:44","slug":"microsoft-complex-event-processing-platform-is-called-streaminsight","status":"publish","type":"post","link":"https:\/\/www.sqlskills.com\/blogs\/bobb\/microsoft-complex-event-processing-platform-is-called-streaminsight\/","title":{"rendered":"Microsoft Complex Event Processing platform is called StreamInsight"},"content":{"rendered":"<p>\nAnd a CTP is <a href=\"http:\/\/www.microsoft.com\/en-us\/download\/default.aspx\" class=\"broken_link\">available now<\/a>.\n<\/p>\n<p>\nI first heard of the .NET Complex Event Processing engine at this year&#39;s TechEd. Until lately, all we had to go on was Torsten Grabs&#39; TechEd presentation and <a href=\"http:\/\/download.microsoft.com\/download\/F\/D\/5\/FD5E855C-D895-45A8-9F3E-110AFADBE51A\/Microsoft%20CEP%20Overview.docx\">a whitepaper<\/a> that gave a fairly high-level overview of its purpose and architecture. And, oh, it was to be released as part of SQL Server 2008 R2. So last week when SQL Server 2008 R2 CTP3 was released, there was also an announcement that a CTP of StreamInsight was available. The CEP engine has a name. And there&#39;s code and libraries&nbsp;to&nbsp;work with.\n<\/p>\n<p>\nDownloaded it last week and installed it. It has no dependencies on SQL Server 2008 R2; in fact it doesn&#39;t even use SQL Server at this point. The metadata is stored in a SQL Server CE database, and the data it&#39;s processing as a stream isn&#39;t particularly stored anywhere unless your event target stores it. There&#39;s an example of consuming data from and storing data in SQL Server, but that&#39;s not always the point.\n<\/p>\n<p>\nAt a quick first glance, it is vaguely reminiscent of SQL Server Notification Services with event sources and event targets, but the similarity ends there. In the Notification Services architecture, events are stored in the database and matched against subscriptions to produce notifications. The focus in SQLNS is on storing events, matching, and formatting notifications. Until the event is stored, it&#39;s not visible to the infrastructure, only to the event provider.\n<\/p>\n<p>\nIn StreamInsight, the focus is on processing and querying the data in the input stream, in real-time. The stream processing architecture is lightweight, and almost all of the use cases presented in the docs name &quot;the ability to handle up to 100000 events per second for a large number of devices&quot; as a goal. Storing events is an option, but the interesting part is being able to query the event stream in-flight and generate output events based on aggregations, outliers, or other characteristics of the stream. In this respect, StreamInsight resembles TIBCO events much more than SQLNS.\n<\/p>\n<p>\nThere is an input adapter architecture and an output adapter architecture. You can write your own event server, use their server infrastructure or imbed it in an existing application. There&#39;s even a model for reporting state changes in the stream (event source) to interested parties in the event sink to implement a pub-sub application. The development model is .NET-based and&nbsp;at the center of it&nbsp;is a LINQ provider over the event stream. You register compiled LINQ queries that execute over to stream and provide selection, projection, joins, aggregation, and some additional operators like timestamp modification. You can also extend the LINQ provider&#39;s functionality with your own user-defined functions.\n<\/p>\n<p>\nRather than continue to regurgitate the documentation (I&#39;m doing this here as an exercise to make sure I &quot;get it&quot;, feel free to disagree with or correct me), I&#39;ll urge you to download the CTP, run the examples, get comfortable with the architecture, and try writing a simple adapter to get a feel for it. There&#39;s a team blog and a forum where the team members hang out (I recognize some of the folks&#39; names there) if you have questions or problems.\n<\/p>\n<p>\nThere&#39;s also a nice event flow debugger to give you visibility over how stream flow through the processing engine, queries, and adapters. But beware&#8230;the debugger needs a later OS like Vista, Windows7, or Windows Server 2008 to work. It looks like the debugger is using event tracing for Windows (ETW) as its trace vehicle, and because it uses the &quot;newer&quot; ETW (ETW-vnext was known once as Crimson, and introduced in the latest OSes) it needs one of these OSes to function.\n<\/p>\n<p>\nSo, happy stream processing!&#8230;<br \/>\nCheers&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>And a CTP is available now. I first heard of the .NET Complex Event Processing engine at this year&#39;s TechEd. Until lately, all we had to go on was Torsten Grabs&#39; TechEd presentation and a whitepaper that gave a fairly high-level overview of its purpose and architecture. And, oh, it was to be released as [&hellip;]<\/p>\n","protected":false},"author":7,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[29],"tags":[],"class_list":["post-632","post","type-post","status-publish","format-standard","hentry","category-sql-server-2008"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v21.9.1 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Microsoft Complex Event Processing platform is called StreamInsight - Bob Beauchemin<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.sqlskills.com\/blogs\/bobb\/microsoft-complex-event-processing-platform-is-called-streaminsight\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Microsoft Complex Event Processing platform is called StreamInsight - Bob Beauchemin\" \/>\n<meta property=\"og:description\" content=\"And a CTP is available now. I first heard of the .NET Complex Event Processing engine at this year&#039;s TechEd. Until lately, all we had to go on was Torsten Grabs&#039; TechEd presentation and a whitepaper that gave a fairly high-level overview of its purpose and architecture. And, oh, it was to be released as [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.sqlskills.com\/blogs\/bobb\/microsoft-complex-event-processing-platform-is-called-streaminsight\/\" \/>\n<meta property=\"og:site_name\" content=\"Bob Beauchemin\" \/>\n<meta property=\"article:published_time\" content=\"2009-08-21T11:45:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2013-01-04T07:59:44+00:00\" \/>\n<meta name=\"author\" content=\"Bob Beauchemin\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Bob Beauchemin\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.sqlskills.com\/blogs\/bobb\/microsoft-complex-event-processing-platform-is-called-streaminsight\/\",\"url\":\"https:\/\/www.sqlskills.com\/blogs\/bobb\/microsoft-complex-event-processing-platform-is-called-streaminsight\/\",\"name\":\"Microsoft Complex Event Processing platform is called StreamInsight - Bob Beauchemin\",\"isPartOf\":{\"@id\":\"https:\/\/www.sqlskills.com\/blogs\/bobb\/#website\"},\"datePublished\":\"2009-08-21T11:45:00+00:00\",\"dateModified\":\"2013-01-04T07:59:44+00:00\",\"author\":{\"@id\":\"https:\/\/www.sqlskills.com\/blogs\/bobb\/#\/schema\/person\/62bfa986c5b5d28fcffd8b4fc409c73e\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.sqlskills.com\/blogs\/bobb\/microsoft-complex-event-processing-platform-is-called-streaminsight\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.sqlskills.com\/blogs\/bobb\/microsoft-complex-event-processing-platform-is-called-streaminsight\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.sqlskills.com\/blogs\/bobb\/microsoft-complex-event-processing-platform-is-called-streaminsight\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.sqlskills.com\/blogs\/bobb\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"SQL Server 2008\",\"item\":\"https:\/\/www.sqlskills.com\/blogs\/bobb\/category\/sql-server-2008\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Microsoft Complex Event Processing platform is called StreamInsight\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.sqlskills.com\/blogs\/bobb\/#website\",\"url\":\"https:\/\/www.sqlskills.com\/blogs\/bobb\/\",\"name\":\"Bob Beauchemin\",\"description\":\"SQL Server Blog\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.sqlskills.com\/blogs\/bobb\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.sqlskills.com\/blogs\/bobb\/#\/schema\/person\/62bfa986c5b5d28fcffd8b4fc409c73e\",\"name\":\"Bob Beauchemin\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.sqlskills.com\/blogs\/bobb\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/6f80e6cc667410857fa6a21931dc528b8092f4d112bf7a8ff7c267674d44ee37?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/6f80e6cc667410857fa6a21931dc528b8092f4d112bf7a8ff7c267674d44ee37?s=96&d=mm&r=g\",\"caption\":\"Bob Beauchemin\"},\"sameAs\":[\"http:\/www.sqlskills.com\/blogs\/bobb\/\"],\"url\":\"https:\/\/www.sqlskills.com\/blogs\/bobb\/author\/bobb\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Microsoft Complex Event Processing platform is called StreamInsight - Bob Beauchemin","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.sqlskills.com\/blogs\/bobb\/microsoft-complex-event-processing-platform-is-called-streaminsight\/","og_locale":"en_US","og_type":"article","og_title":"Microsoft Complex Event Processing platform is called StreamInsight - Bob Beauchemin","og_description":"And a CTP is available now. I first heard of the .NET Complex Event Processing engine at this year&#39;s TechEd. Until lately, all we had to go on was Torsten Grabs&#39; TechEd presentation and a whitepaper that gave a fairly high-level overview of its purpose and architecture. And, oh, it was to be released as [&hellip;]","og_url":"https:\/\/www.sqlskills.com\/blogs\/bobb\/microsoft-complex-event-processing-platform-is-called-streaminsight\/","og_site_name":"Bob Beauchemin","article_published_time":"2009-08-21T11:45:00+00:00","article_modified_time":"2013-01-04T07:59:44+00:00","author":"Bob Beauchemin","twitter_misc":{"Written by":"Bob Beauchemin","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.sqlskills.com\/blogs\/bobb\/microsoft-complex-event-processing-platform-is-called-streaminsight\/","url":"https:\/\/www.sqlskills.com\/blogs\/bobb\/microsoft-complex-event-processing-platform-is-called-streaminsight\/","name":"Microsoft Complex Event Processing platform is called StreamInsight - Bob Beauchemin","isPartOf":{"@id":"https:\/\/www.sqlskills.com\/blogs\/bobb\/#website"},"datePublished":"2009-08-21T11:45:00+00:00","dateModified":"2013-01-04T07:59:44+00:00","author":{"@id":"https:\/\/www.sqlskills.com\/blogs\/bobb\/#\/schema\/person\/62bfa986c5b5d28fcffd8b4fc409c73e"},"breadcrumb":{"@id":"https:\/\/www.sqlskills.com\/blogs\/bobb\/microsoft-complex-event-processing-platform-is-called-streaminsight\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.sqlskills.com\/blogs\/bobb\/microsoft-complex-event-processing-platform-is-called-streaminsight\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.sqlskills.com\/blogs\/bobb\/microsoft-complex-event-processing-platform-is-called-streaminsight\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.sqlskills.com\/blogs\/bobb\/"},{"@type":"ListItem","position":2,"name":"SQL Server 2008","item":"https:\/\/www.sqlskills.com\/blogs\/bobb\/category\/sql-server-2008\/"},{"@type":"ListItem","position":3,"name":"Microsoft Complex Event Processing platform is called StreamInsight"}]},{"@type":"WebSite","@id":"https:\/\/www.sqlskills.com\/blogs\/bobb\/#website","url":"https:\/\/www.sqlskills.com\/blogs\/bobb\/","name":"Bob Beauchemin","description":"SQL Server Blog","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.sqlskills.com\/blogs\/bobb\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.sqlskills.com\/blogs\/bobb\/#\/schema\/person\/62bfa986c5b5d28fcffd8b4fc409c73e","name":"Bob Beauchemin","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.sqlskills.com\/blogs\/bobb\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/6f80e6cc667410857fa6a21931dc528b8092f4d112bf7a8ff7c267674d44ee37?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/6f80e6cc667410857fa6a21931dc528b8092f4d112bf7a8ff7c267674d44ee37?s=96&d=mm&r=g","caption":"Bob Beauchemin"},"sameAs":["http:\/www.sqlskills.com\/blogs\/bobb\/"],"url":"https:\/\/www.sqlskills.com\/blogs\/bobb\/author\/bobb\/"}]}},"_links":{"self":[{"href":"https:\/\/www.sqlskills.com\/blogs\/bobb\/wp-json\/wp\/v2\/posts\/632","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.sqlskills.com\/blogs\/bobb\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.sqlskills.com\/blogs\/bobb\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.sqlskills.com\/blogs\/bobb\/wp-json\/wp\/v2\/users\/7"}],"replies":[{"embeddable":true,"href":"https:\/\/www.sqlskills.com\/blogs\/bobb\/wp-json\/wp\/v2\/comments?post=632"}],"version-history":[{"count":0,"href":"https:\/\/www.sqlskills.com\/blogs\/bobb\/wp-json\/wp\/v2\/posts\/632\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.sqlskills.com\/blogs\/bobb\/wp-json\/wp\/v2\/media?parent=632"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.sqlskills.com\/blogs\/bobb\/wp-json\/wp\/v2\/categories?post=632"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.sqlskills.com\/blogs\/bobb\/wp-json\/wp\/v2\/tags?post=632"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}