Azure On-Premise data gateway

This is the next best thing after Microsoft Azure Relay to connect or expose your on-premise resources to the cloud. In the case of MS Azure Relay  it remove the reliance on opening ports  in the enterprise Firewall to expose WCF endpoints to the external world.

The Azure On-Premise data gateway takes a step ahead and embeds a software into the server that has the database (that the enterprise wishes to expose) .

The good part of the Azure On-Premise data gateway connection is that  there we have the option to select ‘Azure Service Bus’ connection or ‘Non-Azure Service Bus’   connection , albeit with a performance cost attached -since performance suffers with Azure Service Bus connectivity .

The advantages with the Azure Service Bus connectivity is that we have the various options within the service Bus see Service Bus queues, topics, and subscriptions and Relay.The gateway acts as a bridge between the cloud and your on-premises server. Data transfer between the cloud and the gateway is secured through Azure Service Bus. The Service Bus creates a secure channel between the cloud and your on-premises server through an outbound connection on the gateway.

It is recommended that you whitelist the IP addresses, for your data region, in your firewall. You can download the Microsoft Azure Datacenter IP list. This list is updated weekly. The gateway will communicate with Azure Service Bus using the IP address along with the fully qualified domain name (FQDN). If you are forcing the gateway to communicate using HTTPS it will strictly use FQDN only, and no communication will happen using IP addresses.

There still remains the firewall connectivity and configuration that requires to be setup.

sign in to azure

How it works

 

 

 

Azure Service Bus Messaging

https://docs.microsoft.com/en-us/azure/service-bus-messaging/service-bus-messaging-overview

 

Service Bus messaging: flexible data delivery in the cloud

Service Bus is a brokered, or third-party communication mechanism. This is similar to a postal service in the physical world. Postal services make it very easy to send different kinds of letters and packages with a variety of delivery guarantees, anywhere in the world.

Service Bus supports two distinct messaging patterns: Azure Relay and Service Bus Messaging.

The WCF Relay component of Service Bus is a centralized (but highly load-balanced) service that supports a variety of different transport protocols and Web services standards

The relay service supports traditional one-way messaging, request/response messaging, and peer-to-peer messaging. It also

WCF Relay provides many benefits, but requires the server and client to both be online at the same time in order to send and receive messages

 

In contrast to the relay scheme, Service Bus Messaging, or brokered messaging can be thought of as asynchronous, or “temporally decoupled.” Producers (senders) and consumers (receivers) do not have to be online at the same time. The messaging infrastructure reliably stores messages in a “broker” (such as a queue) until the consuming party is ready to receive them

 

The core components of the Service Bus brokered messaging infrastructure are queues, topics, and subscriptions. The primary difference is that topics support publish/subscribe capabilities that can be used for sophisticated content-based routing and delivery logic, including sending to multiple recipients. These components enable new asynchronous messaging scenarios, such as temporal decoupling, publish/subscribe, and load balancing. For more information about these messaging entities, see Service Bus queues, topics, and subscriptions.

Utility for using DataTools Kleber

The DPID is the Australia Post, Delivery Point Identifier. This is an 8 digit number that is unique to every address in Australia, generated by validating against Australia Post’s PAF (Postal Address File).

Requirement : We have to fetch DPID for a bunch of addresses –Shown in the screenshot below.In the below sheet the DPID column is empty and requires to be filled in an automated way(using the utility shown in this blog):

before

Output: We have the DPID entered for the above bunch of addresses –Shown in the screenshot below.It was possible using the tool :

http://kleber.datatools.com.au/

after

To achieve this we have to send to Kleber a  bach of addresses (less than 50 at a time as Kleber can process only 50)

 

  1. The code below first open the Source Excel sheet
  2. Counts how many are there. Breaks them into batches of 20(in this case)
  3. Reads 20 each and sends 20 to kleber to verify the address and fetch the DPID
  4. the results from kleber is concatenated into a single string
  5. Once all the addresses have been verified the application opens the destination excel sheet and writes the result to it.

MVC JTable implementation

My tryst started with creating a simple MVC application for adding and editing data for our cleints.
However we wished to reduce the number of pages the user will have to trverse in the application .Hence
after some research came across JTable a free tool that allows users to expans and contract the area of
vision by clicking on images .Refer to site and ::
Also the editing / adding and deleting of data was possible using modal popup.

All is very easy to setup and forumulate using the demos that are listed in the site ::
However while doing the project from scratch I came across the following challenges and learnings,

Demos dont tell us about linking to a database table.

Updating a single record from the UI.Record instaed of Records

Returning a JSON with var variable.

Nullable datetime values

sending parameters to the Controller

Authorization of users

Combo box with multiple select–add and edit