Palo Alto-headquartered realtime data processing business Hazelcast has gone serverless with a public beta of its Viridian platform.
Hazelcast is that bit that sits in between the firehose of data flooding in from continuous business event pipelines (think IoT or message queues) and spits out actionable context into the likes of databases and alerts.
It’s also scalable – last year Hazelcast Jet hit a billion events per second in a cluster of 45 nodes and 720 vCPUs.
“And there was really no need for us to stop at that point,” Scott McMahon, Hazelcast’s senior director of Solution Architecture, told The Register, “We just figured that was a good round number.”
Things have moved on apace since, and McMahon told us the team were now routinely hitting three billion events per second and were looking to reach the 10 billion mark. “It’s really just, you know, how much does it cost to start up that many AWS instances and run this thing?” he said.
Well, Jeff Bezos’ side projects won’t pay for themselves…
There is also the management overhead of such an undertaking. Enterprise versions of Hazelcast Cloud were once part of since-deprecated Hazelcast In-Memory Data Grid brand, hosted on AWS, Azure or GCP, and, while a simpler proposition than going on-premises, could lack the immediacy and flexibility of something more serverless.
Which is where Viridian comes in, with the self-service provisioning and automatic sizing developers are used to with serverless applications. After all, AWS’s Lambda has quite the following.
Viridian starts with a free-forever tier on sign-up replete with 2GiB of in memory storage (for the time being), a declarative API for building apps that use real-time streaming data, support for streaming SQL and connectors to other cloud data sources (although developers can roll their own if their favored data source is not on the list.) There is WAN replication capabilities and integration with cloud deployments such as Apache Kafka.
It’s all heady stuff, particularly considering the relative grief users face in provisioning the platform for themselves. There are, as with all serverless solutions, downsides. Cost could be an issue for the unwary, although since it is a pay-as-you-use model, there is not the worry of having dozens (or more) unused AWS nodes turning up on an invoice after a developer got a bit too enthusiastic with the corporate credit card.
The beta is closed at present, but should become public shortly. Hazelcast, told us that a move to production was expected “in about two months” based on the feedback the company received during the beta. “There are a few things that we want to observe and optimize,” it added, noting that tweaks might be needed as the service gets loaded up with some “heavy test and development runs.”
The Hazelcast Viridian serverless data platform is currently available on AWS, although is expected to hit GCP before long and Azure in the same timeframe as its release to production. ®