Event‐driven architecture (EDA) is a software model that encompasses the production, reaction, detection, and consumption of events. Event producers can be, for example, an ADLS container, and an event consumer could be an Azure Synapse Analytics pipeline trigger. Therefore, it would be possible to wire up a blob created event to a trigger that executes a pipeline run. Note that it is also possible to create similar functionality with custom events using Event Grid. An event can come in many forms. The Blob Created event is one that happens when a new file is placed onto an ADLS container within an Azure storage account. Similarly, when a blob is deleted, a Blob Deleted event is generated. A consumer plugs into the producer by subscribing to a broker that manages the delivery of events. When you create the broker that will detect these events, you configure the source of the events, i.e., an ADLS container, and specify which events you want to receive, for example, BlobCreated or BlobDeleted. Then, if you want to be notified of these events so that your trigger can react to them, you must subscribe to the broker, which is configured to detect them from the producer. That process of event production, detection, consumption, and reaction is illustrated in Figure 6.39.

FIGURE 6.39 Azure Synapse Analytics custom event notification flow

Azure Event Grid, first introduced in Chapter 3, is the broker used for this kind of behavior. The EDA process begins with the addition of a new file, which was placed into an ADLS container. Once the writing of that file is complete, a Blob Created event is generated. Any subscriber that has signed up to receive this notification will receive metadata that resembles the following:

The format of the event notification sent to the subscribers is in JSON. This means that the service that receives the notification must be able to receive and parse the JSON data. There are numerous sources that can handle these event notifications. As shown in Figure 6.39, two sources are an Azure function and a webhook. An Azure function is a serverless compute offering designed to provide quick access to compute, on demand, to process small snippets of code. Therefore, an Azure function is a very good option for implementing this kind of process. A webhook basically provides the ability to route the event data to any endpoint that supports HTTPS. This can be an Azure App Service or an on‐premises application.

An Event Grid subscription is a management utility that can contain multiple Azure Event Grid topics. Consider that the Event Grid topic is the actual endpoint with which subscribers bind to receive events. When you have numerous EDA solutions that contain many event notification flows, an Event Grid subscription is used to manage all your Event Grid topics. You use the features available from an Event Grid subscription to manage all your EDA services. Once you have an Event Grid topic, you add it to an Event Grid subscription. Then, from the service that wants the event notification, you add the service to the Event Grid topic. To bind the Event Grid topic to an Azure Synapse Analytics pipeline, you create a custom event and select Custom Events from the Type drop‐down text box. You add the Event Grid topic into the Event Grid Topic Name text box, as shown in Figure 6.40.

FIGURE 6.40 Azure Synapse Analytics custom event trigger

The configuration of the custom event also gives you the option to filter the events based on the subject and the event type. If you look at the JSON example provided previously, you will see an attribute named subject. If the subject contained in the event metadata includes, for example, brainjammer, then the pipeline will be triggered, so long as the event type matches as well. In addition to the subject attribute, there is also an eventType attribute in the JSON event metadata. The configuration limits the triggering of the pipeline to event types of either BlobCreated or BlobDeleted. Only when both the subject and eventType are matched will the trigger fire and the pipeline run. The filter on subject is optional; however, you must add at least one event type.