Skip to content

Latest commit

 

History

History
173 lines (106 loc) · 14.2 KB

README.md

File metadata and controls

173 lines (106 loc) · 14.2 KB

Sync NCache with PostgreSQL through Notifications

Using NotifyExtensibleDependency and Read-Through

Table of contents

Introduction

This project highlights the NCache NotifyExtensibleDependency, ReadThruProvider, Resync feature and how it can be used to synchronize NCache with a PostgreSQL database. NotifyExtensibleDependency is a cache synchronization strategy for tackling the stale data problem, to be discussed shortly. Its primary purpose is to give the solution architect the flexibility to integrate a real-time customized logic that monitors and processes datastore change notifications directly into the cache hosting processes running on the cache servers.

ReadThruProvider will use your custom read-through provider to communicate with the back-end data source. Using this you can incorporate your custom logic to load data from the configured data source. In Read-Through Caching, NCache will call your provider to load data from the data source behind the get call, in case of a cache miss.

In this scheme, the developer deploys a provider(NotifiyExtensibleDependencyProvider and ReadThruprovider) containing the datastore state-change monitoring and processing logic behind the depedency along with the lifecycle hooks into the NCache servers. The servers then invoke the dependency-related methods which encapsulate the custom logic and determine whether or not to remove the associated cached item. The main strengths of this feature are as follows:

  • Stale Data Problem Mitigation

    Although using NCache with PostgreSQL boosts application performance, there is one issue that needs to be kept in mind. When you start caching with a primary data store such as PostgreSQL, two copies of the same data will be created, one in the primary data store and the other in the cache. Any direct update to the database data could render the cache data stale. With NotifyExtensibleDependency, not only are we taking advantage of the increased read performance provided by NCache, but we can also make sure that stale data does not persist in the cache.

    If you have configured the read thru provider you can choose to check the rsync options to true with the NotifyExtensibleDependency so when the object get invalidated or expired NCache automatically loads the data from the database using the configured ReadThruProvider.

  • Native NCache API Support

    With support built into the NCache core logic, NotifyExtensibleDependency extends the power NCache provides when it comes to high availability, reliability and scalability and adds the mechanisms which ensure that cached-data fully agrees with the primary datastore state at all times.

The following are the advantages of NCache together with this feature when used with PostgreSQL:

  • Faster Read Operations

    Using NCache as your distributed caching solution, application performance is improved since it is an in-memory key-value store which greatly improves data read performance.

  • Improved Scalability

    Using NotifyExtensibleDependency, all the cache synchronization operations are handed over to the clustered cache itself, allowing the clients to focus on the core business logic.

    Not only does this create a clean logical separation of concerns among the NCache client and servers but it also provides improved scalability of the overall system architecture since any increase in notifications load can easily be handled by scaling out the NCache cluster independently of the client-side infrastructure, an important implication for today's cloud-based microservices applications.

Postgresql Asynchronous Notifications Support For NCache Synchronisation

The PostgreSQL Notifications system is based on the LISTEN/NOTIFY mechanism and is similar to the Pub/Sub Model.

The client (in our case NCache) registers with a specific channel by calling the LISTEN command within the NotifyExtensibleDependency Initialize method and then the PostgreSQL connection is put in a wait state by invoking the NpgsqlConnection Wait call.

On the PostgreSQL database side, you need a trigger that invokes a trigger function. By specifying that the trigger should be called AFTER UPDATE and AFTER DELETE, the trigger function sends a JSON payload on a channel.

Any clients LISTENING on the channel using the previous LISTEN command get the payload and processes it using the information given therein. In our case, we take the payload and if the data matches the information about the cached item the dependency is attached to, we invoke the DependencyChanged event handler.

Sample Application Structure

A sample NotifyExtensibleDependency implementation given in the sample follows the approach given in the previous section and you can view the source code here.

A sample ReadThruProvider implementation given in the sample follows the approach given in the previous section and you can view the source code here.

Also given in this project are the scripts for creating the demo customers table, the trigger to invoke the trigger function for notification and the trigger function itself.

Besides these, there are two console applications that reference the NotifyExtensibleDependency .NET Standard 2.0 library. One is PostGresDependencyNETConsoleUI which is a .NET Framework 4.7.2 application and the other is PostGresDependencyNETCOREConsoleUI which is a .NET Core 3.1 application.

These two console applications are identical in their source code and are meant to show that both a .NET Core client and a .NET framework client can make use of the same NotifyExtensibleDependency feature.

Besides these, there are two console applications that reference the ReadThruProvider .NET Standard 2.0 library. One is NETConsoleUI which is a .NET Framework 4.7.2 application and the other is NETCOREConsoleUI which is a .NET Core 3.1 application. These two console applications are identical in their source code and are meant to show that both a .NET Core client and a .NET framework client can get the value from datasource automatically and then from cache using ReadThruProvider feature.

Prerequisites

Before the sample application is executed make sure that:

  • The latest version of NCache Enterprise is installed. The latest version can be gotten from here. If you are using a Linux cache server, download the Enterprise .NET Core installation. For a Windows cache server, you can install either the .NET Framework or the .NET Core installation.

    NOTE: The rest of this article assumes you have installed the .NET Framework installation although the steps for the .NET Core application are exactly the same.

  • Make sure the relevant ports are open to allow for full communication between NCache and the PostgreSQL Database. This sample was tested with PostgreSQL version "10.15"

  • In your PostgreSQL database, run the SQL CREATE scripts given in the Resources folder to create the customers table, the customers table trigger that will be invoking the Notifications trigger function, and the trigger function itself.

    IMPORTANT:Confirm that the trigger is enabled.

  • In both the PostGresDependencyNETConsoleUI and PostGresDependencyNETCOREConsoleUI applications, change the Program.cs source code by adding the connection string to the PostgreSQL database that contains the customers table and the associate trigger and trigger function as shown below:

    // Beginning of Program.cs

// Change the connection string to your PostGreSQL Database
  const string connectionString = "Host=localhost;Port=5432;Username=postgres;Password=password;Database=database";

// Rest of code

    Furthermore, make sure the client.ncconf is configured with the IP address of one of the servers of your demo cache in the server attribute as shown below:

    <?xml version="1.0" encoding="UTF-8"?>
<configuration>
 <ncache-server 
 	connection-retries="5" 
 	retry-connection-delay="0" 
 	retry-interval="1" 
 	command-retries="3" 
 	command-retry-interval="0.1" 
 	client-request-timeout="90" 
 	connection-timeout="5" port="9800"/>
 <cache 
 	id="democache" 
 	client-cache-id="" 
 	client-cache-syncmode="optimistic" 
 	default-readthru-provider="" 
 	default-writethru-provider="" 
 	load-balance="True"
 	enable-client-logs="False" 
 	log-level="error">
   	<server 
 		name="***.***.***.***"/>
 </cache>
</configuration>

    After making the above mentioned changes, build the solution in Release mode to generate the assemblies.

  • Create a cache named democache using the NCache Web Manager GUI. The steps for doing this are given here.

    IMPORTANT: Do not start the cache after creating it as we will be deploying the NotificyExtensibleDependency implementation dependencies as shown in the next step.

  • Once democache is created configure the ReadThruProvider using the tutorial mentioned here. Name the ReadThruProvider with Name "PostGreSqlReadThruProvider" and add connectionSting as parameter with "Host=localhost;Port=5432;Username=postgres;Password=password;Database=database".

  • Once democache is created, we need to deploy the assemblies related to PostGreSQLNotificationDependency and PostGreSQLBackingSourceProvider to allow for the PostgreSQL notification logic and ReadThruProvider with PostGreSQL to run on the server side onto democache.

    Since we are assuming an NCache Enterprise .NET Framework server installation, we will deploying the .NET Framework assemblies. For that, use the assemblies given in the /bin/release folder for the PostGresDependencyNETConsoleUI application.

    Once the assemblies are deployed using the steps given here, start the cache.

  • On the database side, run an INSERT script on the customers table to insert a row with customerid field set to ALFKI. We will applying UPDATE and DELETE calls on this field to demonstrate real-time cache-invalidation on the NCache side using the NotifyExtensibleDependency implementation deployed with the assemblies during the previous step.

Build and Run the Sample

  • Run the PostGresDependencyNETConsoleUI application. This is will insert a dummy cache item in the cache with dependency set on it. This dependency is an instance of the NotifyExtensibleDependency implementation as shown here.

  • In the Statistics page of the NCache Web Manager for democache, the Count field should now be 1, signifying that the item was successfully added along with the dependency on the PostgreSQL database.

  • On the database side, run either an UPDATE or a DELETE command script on the customers table for row with customerid field set to ALFKI. Once the transaction is complete, NCache will receive a notification from PostgreSQL using NOTIFY command within the trigger function that we created.

    In turn, the cached item will be invalidated and removed from cache, as demonstrated by the Count field in the NCache Web Manager Statistics page for democache going back to 0.

  • Repeat the steps for the PostGresDependencyNETCoreConsoleUI application to confirm that the same behavior is observed when data is added from a .NET Core Client application onto an NCache Enterprise .NET Framework application.

Additional Resources

Documentation

The complete online documentation for NCache is available at: http://www.alachisoft.com/resources/docs/#ncache

Programmers' Guide

The complete programmers guide of NCache is available at: http://www.alachisoft.com/resources/docs/ncache/prog-guide/

Technical Support

Alachisoft © provides various sources of technical support.

Copyrights

© 2020 Alachisoft