Thanks to Mexia’s generosity I was able to attend the INTEGRATE 2017 event in London last week. With 28 international speakers from both Microsoft and the MVP community, this was unquestionably the biggest integration summit of the year. This time, I not only had the opportunity to attend but was also one of the invited speakers as well – a huge honour and thrill. As always, it was a great pleasure to catch up with my integration friends from around the globe, but it was also inspiring to hear firsthand from Microsoft about the advances they have made and are making within the technology space that enables teams like Mexia to deliver the best integration solutions. This post will attempt to summarise the messages that Microsoft delivered at this event, whilst a forthcoming post will cover topics presented by the community MVPs.

Keynote

clip_image001After a brief welcome by BizTalk360 CEO and event organiser Saravana Kumar (during which he announced the 2nd INTEGRATE of the year to take place this October in Redmond, USA), Principal Group Program Manager Jim Harrer gave an inspiring talk that summarised Microsoft’s vision for integration. Alluding to requests from some community members to “slow down”, he made it unapologetically clear that the Pro Integration team would continue full steam ahead in delivering constant advances and improvements; it’s up to the rest of us to keep up! Last year’s motto was “BizTalk & Logic Apps – Better Together”, but this year the toolkit has expanded to include so much more with API Management, Service Bus, Functions, and DB2, as well as tools for gathering insights such as Text Analytics, Computer Vision and Face API.

With this array of capabilities and all of the other services that Microsoft Azure has to offer, there are limitless opportunities to “integrate at the speed of business” and proactively provide solutions that otherwise seem impossible. Jim  summarised the key tips for maximising application value:

  • Look for opportunities to improve customer experience
  • Make it easier for prospects to become customers
  • Seek integration opportunities to yield competitive advantage
  • Use Insights to gain business efficiency

Jim was also joined by his team members Jeff Hollan, Jon Fancey, and Tord Glad Nordahl who each gave previews of the topics they would cover in depth in later sessions. They also introduced the fictitious “Contoso Fitness” business case that would become the scenario for all of their demos during the event.

“Serverless” Computing: Logic Apps & Functions

Featured in no less than seven sessions (five by Microsoft), Logic Apps was undeniably the most celebrated tech of this event. And with all the advances made recently, it is little wonder that the Pro Integration team would want to spend the time highlighting their achievements. Principal Program Manager Kevin Lam kicked off the first afternoon session on Day 1 with an overview of all of Logic Apps capabilities, including triggers, actions, integration accounts, security, DevOps capabilities, and the 160+(!) managed connectors that are currently available. Senior Program Manager Jeff Hollan joined Kevin to perform a demonstration of how the workflow capabilities of Logic Apps gave a backbone to an IoT solution for improving premier support for customers. The solution was able to orchestrate reports of equipment telemetry and user-initiated complaints, integrating with an Asset Management System and an Incident Management System, as well as using Insights and Analytics to proactively determine when equipment would require either maintenance or replacement.

It was also thrilling to see the list of features that are coming very soon, including:

  • UK geo-region
  • Expression authoring & Intellisense
  • Expression tracing
  • Advanced scheduling
  • Large messages
  • Drafts
  • Mock Testing
  • Resubmit from a failed action
  • Tracking & monitoring across Logic Apps in OMS
  • Batch

The improvements in expression authoring and tracing will lower the barrier even more by making it easier to wield the sometimes baffling Workflow Definition Language. In fact, this may even reduce the need to call out to an Azure Function in some cases. And the resubmit at a failed action essentially gives us the suspend & resume capability that we’ve all come to know and love in BizTalk Server. These are dramatic improvements that we can look forward to emerging within the next few months.

In another session on Day 2, Jeff Hollan and Program Manager Derek Li gave us a peek in the “black box” of Logic Apps by revealing the internal workings of the engine. This was truly an epiphany for some as Jeff explained how loops are not necessarily sequential but are processed in batches of up to twenty parallel actions. This is part of what makes Logic Apps so massively scalable. He also explained the “at least once” principle and the importance of understanding that an action may be executed twice in some cases. However, he was able to boast that in two years, there has never been a run that did not finish executing!

Derek went on to show a demo using Logic Apps for shipment notifications. This highlighted the use of scopes for exception management and controlling the flow using the “run after” directive:

exception handling

(image by Srinivasa Mahendrakar)

Other gems that Derek showed were the settings that allow you to override the default retry behaviour (4 attempts at 20 second intervals) and a “Single Instance” flag that ensures a recurrence trigger will not fire if the previously triggered instance is still running. Finally, Jeff wrapped up with an explanation of Workflow Definition Language syntax, emphasizing that expressions are evaluated from the “inside out” and providing a clear listing of the symbols and their meanings:

@ – used to indicate an expression (only if 1st char in the string), escaped by double (@@)

() – encapsulates the expression parameters, e.g. @length(‘Hello world’)

{} – means “string”, same as @string(), e.g. “stringify”

[] – Used to parse a JSON object, but is NOT XPath ( use @xpath instead )

clip_image001

In addition to Logic Apps, Jeff Hollan also conducted a session on Azure Functions. He started off by explaining what “serverless” actually means in terms of Logic Apps and Functions, including the benefits of event-driven scalability and micro-billing. Functions also provide benefits of reduced DevOps, ability to focus on the business logic and not the plumbing, and the reduced time to market. By installing the Azure Functions tools in Visual Studio 2017, you can now build locally and gain the benefits of unit testing, Intellisense, and breakpoint debugging! And your locally running functions can still be triggered by Azure resources, e.g Service Bus queues/topics.

One thing is clear – if you’re using the consumption based model, you need to be able to deal with latency as there is typically a “warm-up” interval upon invocation.

Both Logic Apps and Functions were also featured in a presentation by MSIT to show how they are migrating their internal processes to iPaaS – but more about that later.

Messaging Yesterday, Today, & Tomorrow

This was the title of the session delivered on Day 1 by Principal Program Manager Dan Rosanova. Despite feeling under the weather, Dan’s usual dry wit and flare was in full force as he took us on a journey through the history of messsaging, all the way from Teknetron Information Bus through MSMQ, TIBCO, BizTalk, RabbitMQ, Azure Service Bus and finally Event Hubs. The numbers have always been staggering, but the growth in one year from 2.75 trillion Event Hubs messages per month to the present 23 trillion messages per month is nothing short of astounding. Even more staggering is how many failures are associated with a five 9’s SLA when we’re talking numbers this big:

clip_image001[4]

clip_image001[6]Dan talked about the evolving messaging landscape and how Microsoft is well-placed in the four quadrants defined by Bi-directional Socket Relays, Logging & Big Data Streaming, “Enterprise” Messaging and Simple Messaging. He also mentioned the shift in both simple and enterprise messaging from on-premises to the cloud. Azure Relay was also a major feature of his talk, described as “the basis of Hybrid Cloud” and including a great example of how a hybrid relay binding eventually resolves to a direct socket connection. (This section was a really good precursor to my talk on Hybrid Connectivity two days later.)

Dan finally wrapped up with a description of the new GeoDR service and how it allows you to explicitly fail-over your Service Bus connection strings to another region. Lastly, he left us with a bit of a teaser for something to come…  “What if messaging just happened automatically?”

BizTalk Server – Alive & Well

Whilst Azure related topics dominated the agenda at INTEGRATE, there were still some notable presentations on BizTalk Server peppered throughout. Two of these were delivered by Program Manager Tord Glad Nordahl who reminded us of the power of BizTalk to connect anything, whether on-prem or in the cloud. He also discussed the new capabilities of BizTalk Server 2016 (which is the only existing version of the product that will still be in support one year from now):

  • Logic App Adapter
  • High Availability in IaaS (via SQL Server AlwaysOn)
  • Drummond Certification
  • Ordered Delivery for Dynamic Ports
  • Adapter improvements (SAP, FTP, File)

Tord also discussed the Feature Pack 1, including a whole deep-dive session dedicated to it on Day 3. This pack (supported only on Enterprise Edition with Software Assurance) has had only one minor bug discovered since its release! Nevertheless it is packed with features (hmmm… is that why they call it a “Feature Pack”?) that make a huge difference to the overall product:

  • Automatic Deployment
    • VSTS deployments
    • Continuous Integration
    • Token support in binding files
  • Management API
    • Exposes ExplorerOM endpoints via REST
    • Enables Remote Management
    • Anything you can do in the Admin console is now exposed!
  • Application Insights
    • Tracking data to Insights
  • Operational Data
    • Pump tracking data thru Power BI
  • Advanced Scheduling
    • Monthly
    • Weekly
    • Daily (i.e. every last Friday of the month)
    • Timezone support for port service windows
  • Always Encrypted supported

Citing an aggressive release cadence, it is now as clear as ever that BizTalk Server is here to stay as the product team continues its devotion to constant refinements:

clip_image001[8]

On the last day, Tord was also able to announce the immediate release of the BizTalk Server Migration Tool, a utility for migrating your BizTalk environment from one environment to another, supporting backups and upgrades in version.

Other presentations surrounding BizTalk Server included a performance optimisation session by Sandro Pereira, a health check discussion by Saffieldin Ali, and a talk on Hybrid Integration by Wagner Silveira. Also, Saravana Kumar gave a talk on environment management using BizTalk360.

APIs, Events, and Flow

Other topics well presented by Microsoft at INTEGRATE included:

  • clip_image001[22]Event Hubs – Program Manager Shubha Vijayasarathy gave a detailed talk on Event Hubs, described as a “distributed streaming platform-as-a-service” that is ideally matched for handling Big Data with its hyper-scale, low-latency pub-sub capabilities. Shubha also announced the General Availability of Event Capture, which simplifies both archiving and batch processing. This service can automatically send your data to a chosen destination without the reliance on any other data transfer service. Although already GA, the price change will not go into effect until 1st August 2017. Shubha talked about the conceptual architecture or Event Hubs, explaining how partitions work, and showed a very helpful slide outlining the difference between Event Hubs and Service Bus Topics. She also talked about scaling capabilities, auto-inflation (coming later this month), and pricing – as well as reminding us of the importance of the namespace updates. What can we expect to see soon? GeoDR capability, Encryption at Rest, and metrics in the new portal.
  • Flow – Program Manager Derek Li conducted a session on Microsoft Flow, the “last mile of integration”. Dubbed as a “citizen integrator” tool, Flow helps non-developers create their own automated workflows to assist in otherwise mundane tasks like receiving notification, copying files, collecting data, and automating approvals. A scaled-down version of Logic Apps which uses the same connectors, Flow provides a self-service adhoc capability for creating simple integrations that would otherwise steal time from bigger and more critical development activities by integration professionals. Proving how easy it is to get up and running, Derek performed a live demo of creating a simple solution for emailing a daily class schedule to a Contoso Fitness trainer.
  • IBM Systems Integration – Principal Program Manager Paul Larsen and MVP Steve Melan gave a presentation on integrating heritage IBM systems with both the cloud and on-premises connectors. It quickly became apparent that the term “heritage” is synonymous with “legacy” – only with a more positive slant. Paul showed some interesting photos of mainframe systems from 1947 to present day before launching into a discussion about how the DB2 Connector and the On-Premises Data Gateway help to link Azure systems with on-prem mainframes.
  • API Management – Principal Program Manager Vlad Vinogradsky and Senior Program Manager Matthew Farmer talked about the “State of Azure API Management”. According to Gartner, “APIs make digital society and digital business work; they are the basis of every digital strategy.” API Management is a full-featured hub for custom APIs, supporting recent new features like SOAP to REST, Internal VNETs, API Mocking, Swagger editing, AAD B2C integration and more. Vlad talked about the difference between “versions” and “revisions” and how APIM supports both. Matthew then demonstrated how versioning works with a simple fitness calculator update. Finally, Vlad finished with roadmap slide for the rest of this year, including ARM templates support, capacity metrics in Azure Monitor, Key Vault integration, payload validation, and much more.
    clip_image001[17]  image

Microsoft Bets on Its Own Tech

Jim Harrer made the comment more than once that “Microsoft eats its own dogfood” – and that statement was proven in the presentation by Program Manager Mayank Sharma and Divya Swarnkar, members of the EPS Integration Team at Microsoft.  In this session, they explained how MSIT is shifting their internal systems from BizTalk Services to iPaaS, using an extremely well-architected solution comprised of Logic Apps, API Management, Azure Functions, OMS, and Integration Accounts. The solution supports their digital supply chain, running trade integrations and all customs declaration transactions using both EDI and non-EDI protocols. They will be ready to retire their BizTalk Services instances by the end of this month. Divya went on to explain how they can test the modifications to their solution in production by creating a separate version branch and using APIM to direct certain transactions down the new path. They’ve also established a robust business continuity by deploying the processing pipeline to a secondary region and keeping the Integration Accounts synchronised with the primary region via Logic Apps. This would enable a rapid failover if necessary.

clip_image001[24]   clip_image001[26]

It’s wonderful to see that Microsoft has enough confidence in their own products that they would pin their critical business systems on it.  With over 1000 partners and 170+ million messages per month, it’s certainly not a trivial integration landscape! Their own reliance on Microsoft integration tech will become a flagship for other organisations. A whitepaper about this solution is due out this month.

Summary

Large events like INTEGRATE are always a great vehicle for Microsoft to reveal the progress it is making in the digital transformation sector, as well as introduce the latest advances. This event was no exception, and I believe that every participant walked away with even greater enthusiasm and confidence in Microsoft’s ability to address the integration challenges of today. As Jim Harrer stated in the keynote, our roles as professional integrators are changing – we are becoming the “Can you do this?” people as we continue to astonish our business stakeholders with the rapid implementation of solutions and  the ability to identify opportunities for adding business value – all thanks to the constant innovations from Microsoft.

4 responses to “Microsoft’s Messages from INTEGRATE 2017”

  1. […] already published one blog post summarising my take on the messages delivered by Microsoft (which accounted for half of the […]

  2. […] my previous post, I talked about the messages conveyed by the Microsoft product team at the INTEGRATE 2017 event in […]

Leave a Reply