C2B2 logo icon

Investigating Infinispan: Can remote and local cache retrieval coexist?

c2b2 JBoss consultant, Alessandro Vaccarezza, troubleshoots an application deployment issue for a customer’s WildFly environment, and finds the solution in an Infinispan cache configuration change.

We recently came across an intricate issue involving Infinispan, the distributed in-memory key/value data store technology that sits under the hood of JBoss EAP and WildFly. In this deployment, the customer application was on a WildFly environment, and using a remote instance of Infinispan 9.1.0, running in replication and embedded mode. 

The Goal

Their development team goal was to create a few cache entries programmatically at application deploy, whilst at the same time being able to retrieve them via remote Hot Rod Clients. We implemented a listener for this, but encountered some retrieval issues.

Here’s a simplified version of the listener:

....
import org.infinispan.configuration.cache.CacheMode;
import org.infinispan.configuration.cache.Configuration;
import org.infinispan.configuration.cache.ConfigurationBuilder;
import org.infinispan.manager.EmbeddedCacheManager;
import org.infinispan.Cache;

import uk.co.c2b2.beans.SamplePOJO;

public class SomeServlet extends HttpServlet {

	@Resource(lookup = "java:jboss/infinispan/container/clustered")
	private EmbeddedCacheManager manager;

	public void init(ServletConfig aServeletConfig) throws ServletException {
		// ..........
		start();
	}

	private void start() {
		Map<String, SamplePOJO> cache = new HashMap<>();

		cache.put("key1", new SamplePOJO("1", "value1"));
		cache.put("key2", new SamplePOJO("2", "value2"));
		cache.put("key3", new SamplePOJO("3", "value3"));
		cache.put("key4", new SamplePOJO("4", "value4"));
		cache.put("key5", new SamplePOJO("5", "value5"));

		performOperationsOnCache("SampleCache", cache);
	}

	protected Configuration getCacheManagerConfig() {
		Configuration config = new ConfigurationBuilder().clustering().cacheMode(CacheMode.REPL_SYNC)
				.invocationBatching().enable().storeAsBinary().build();

		return config;
	}

	public <K, V> AdvancedCache<K, V> performOperationsOnCache(String name, Map<K, V> map) {

		if (!manager.cacheExists(name)) {
			manager.defineConfiguration(name, getCacheManagerConfig());
		}

		Cache<K, V> currentCache = manager.getCache(name);

		currentCache.putAll(map);

		SamplePOJO element3 = (SamplePOJO) currentCache.get("key3");

		// do something with element3
	}

}

As you can see, the code is pretty straightforward; you should note the method getCacheManagerConfig(), which defines the runtime configuration settings for the EmbeddedCacheManager.

What we encountered...

The start() method builds a Map of entries, and the calls the performOperationsOnCache() method. SamplePOJO is, unsurprisingly, a POJO. However the issues we discovered persisted, regardless of whether the POJO was simple (with just primitives) or a more complex one. 

The following things happened within performOperationsOnCache():

  1. the current cache was retrieved
  2. the map with the new cache entries was added to the current cache
  3. a newly inserted cache entry was retrieved 

However, the first - and most evident - problem that arose was the impossibility to fetch newly inserted cache entries via Hot Rod Clients. 

Embedded Mode vs. Compatability Mode

We realised that this was due to the embedded mode being enabled for Infinispan; and so switched to compatibility mode.

The Configuration object declaration and initialization then becomes:

Configuration config = new ConfigurationBuilder().clustering() .cacheMode(CacheMode.REPL_SYNC) 
.compatibility() .enable() .invocationBatching() .enable() .storeAsBinary() .build();

However, despite this change, the issue was far from being resolved. In fact, while the cache could be accessed remotely, the same didn’t happen when there was a local access.

When retrieving one of the newly inserted cache entries... 

(SamplePOJO element3 = (SamplePOJO) currentCache.get("key3"))

... a ClassCastException was thrown. However, it was clear that the casting didn’t work, as the value was fetched as a byte{}, and that was it!

Marshalling Mechanisms

So, we decided that our next step was to do some investigation of Infinispan marshalling mechanisms. We initially tried solving it by updating the marshaller to an instance of GenericJBossMarshaller, but didn’t get the results we expected:

Configuration config = new ConfigurationBuilder().clustering() .cacheMode(CacheMode.REPL_SYNC) .compatibility() .enable() .invocationBatching() .enable() .marshaller(new GenericJBossMarshaller()) .storeAsBinary() .build();

A few other tunings were tried, but still the situation didn’t change.

User Defined POJOs

We then had an idea! What if the issue was similar to one we had encountered with Infinispan 9.0, when compatibility mode didn’t support user defined POJOs?

This had occurred when objects were stored in an un-serialised fashion, resulting in Infinispan being unaware of these classes. We investigated this, and were prepared to apply a solid workaround, when after more research we realised that this couldn’t be the problem, because that issue had been fixed in Infinispan 9.1.0! 

Having eliminated a series of potential causes, we went back to the drawing board, and eventually found the root of the problem. 

Solution!

The cache manager we used to create the cache with, was also used by the Infinispan server. This cache was internally being created with an encoding scheme suitable to be used by remote clients -  meaning the storage mechanism accepts POJOs as values, but reads/writes are done using byte[].

Problem discovered now the only thing we had to do was to fix it!

All we needed to do was specify that no encoding was needed, which was accomplished by wrapping the cache using the IdentityEncoder class:

Cache cache = cache.getAdvancedCache().withEncoding(IdentityEncoder.class)

Adjusted code:

...
import org.infinispan.configuration.cache.CacheMode;
import org.infinispan.configuration.cache.Configuration;
import org.infinispan.configuration.cache.ConfigurationBuilder;
import org.infinispan.manager.EmbeddedCacheManager;
import org.infinispan.Cache;

import uk.co.c2b2.beans.SamplePOJO;

import org.infinispan.AdvancedCache;
import org.infinispan.commons.dataconversion.IdentityEncoder;

public class SomeServlet extends HttpServlet {

	@Resource(lookup = "java:jboss/infinispan/container/clustered")
	private EmbeddedCacheManager manager;

	public void init(ServletConfig aServeletConfig) throws ServletException {
		// ..........
		start();
	}

	private void start() {
		Map<String, SamplePOJO> cache = new HashMap<>();

		cache.put("key1", new SamplePOJO("1", "value1"));
		cache.put("key2", new SamplePOJO("2", "value2"));
		cache.put("key3", new SamplePOJO("3", "value3"));
		cache.put("key4", new SamplePOJO("4", "value4"));
		cache.put("key5", new SamplePOJO("5", "value5"));

		performOperationsOnCache("SampleCache", cache);
	}

	protected Configuration getCacheManagerConfig() {
		Configuration config = new ConfigurationBuilder().clustering().cacheMode(CacheMode.REPL_SYNC)
				.invocationBatching().enable().storeAsBinary().build();

		return config;
	}

	public <K, V> AdvancedCache<K, V> performOperationsOnCache(String name, Map<K, V> map) {

		if (!manager.cacheExists(name)) {
			manager.defineConfiguration(name, getCacheManagerConfig());
		}

		Cache<K, V> currentCache = manager.getCache(name);

		AdvancedCache<K, V> currentAdvancedCache = (AdvancedCache<K, V>) currentCache.getAdvancedCache()
				.withEncoding(IdentityEncoder.class);

		currentAdvancedCache.putAll(map);

		SamplePOJO element3 = (SamplePOJO) currentAdvancedCache.get("key3");

		// do something with element3
	}

}

So, despite vague and occasionally unclear documentation, the problem was identified and fixed, ensuring that the client’s application deployment goals were reached using Infinispan 9.1. These issues were overcome with a little investigation, and Infinispan,a solid and mature product, performs well and is definitely suitable for production use.