During its Build 2020 online developer conference this week, Microsoft announced a number of new and updated products targeting enterprises engaged in big data analytics. The newest product in the company’s portfolio is Azure Synapse Link, which enables data analytics on live operational data and will debut in the coming weeks alongside new Azure Synapse features in preview and Project Cortex. Microsoft will also soon launch autoscale-provisioned throughput for Azure Cosmos DB (which was previously called Autopilot) in general availability, where it will be joined by a small-footprint database engine for edge devices called Azure SQL Edge.
As of 2017, big data had been adopted by 53% of companies, according to a survey conducted by Dresner Advisory Services. The benefits can be substantial, with early adopters reporting 8% to 10% increases in profit and 10% reductions in overall costs. But it’s a challenge making big data standard practice; a majority of organizations peg inadequate know-how as the cause of implementation delays and failures.
Microsoft’s tools aim to automate some of the most arduous big data processes while enabling customers to tailor parameters to various use cases.
For instance, Project Cortex, which will become generally available this summer, leverages AI to automatically classify an organization’s content. It analyzes documents, conversations, meetings, and videos to identify domain experts as it populates a knowledge database, and it surfaces info through cards and pages embedded in Office apps, Outlook, and Microsoft Teams. Through developer APIs, managed metadata services (MMS) in Microsoft Graph, and new integration with Language Understanding service in Azure, Microsoft says the more than 75 organizations using Cortex in private preview (including Arla Foods and Siemens Healthineers) have delivered knowledge from millions of documents and videos.
For customers with hundreds, thousands, or tens of thousands of internet of things (IoT) devices to manage, Azure SQL Edge migrates many of the capabilities offered by Microsoft’s Azure SQL Database and SQL server to hardware-constrained devices. The small-footprint container (less than 350MB) supports Arm and x64-based edge gateways and machines while offering low-latency analytics that combine streaming and time-series data, as well as in-database AI, machine learning, and graph data. SQL Edge’s other spotlight features include policies that protect data at rest and in motion, a programming surface in common with Azure SQL Database and SQL Server, and support for cloud-connected and fully decentralized edge scenarios.
Azure Synapse Link, which is now available with Azure Cosmos DB (and soon with Azure SQL, Azure Database for PostgreSQL, and Azure Database for MySQL), is a cloud-native implementation of the HTAP (hybrid transactional analytical processing) paradigm that bridges the gap between transaction processing and analytics. It’s designed to enable customers to gain insights through live operational data in Azure Synapse Analytics without the need to extract, transform, or load steps. This real-time data can then be combined with existing analytical stores to gain a holistic view of a business.
When the complementary Azure Cosmos DB autoscale provisioned throughput service launches in summer 2020, it will offer “single-digit” millisecond latency and a claimed 99.999% availability while automating planning and management of throughput capacity. After customers select a max level for autoscale provision throughput, Azure Cosmos DB will scale provisioned throughput up and down based on the workload usage, in theory relieving engineers of the need to manage it themselves.
In related news, Microsoft today revealed new capabilities coming to its Azure Synapse limitless analytics product, including a unified experience within Synapse Studio, plus integrated SQL and Spark engines, serverless data lake exploration with Synapse SQL, shared metadata store, and built-in Power BI authoring.
Azure Synapse can query data at “petabyte-scale,” directed by lines of SQL, with intelligent workload management and concurrency features that optimize the performance of queries in real time. Synapse Studio provides tools for data prep, data management, data warehousing, big data, and AI tasks, letting users manage data pipelines and build proofs of concept. And on the security side of the equation, Azure Synapse features automated threat detection and always-on data encryption, with fine-grained access controls and column- and row-level security, in addition to dynamic data masking.