Real-time platform migration, designed for zero downtime
A big-data platform moved from AWS to Google Cloud while it ran
The client needed its streaming and analytics platform moved from AWS to Google Cloud in real time without disrupting service. We moved ~120 TB, re-platformed onto Kubernetes and Dataproc, and upgraded the core stack on the way.
~90 TBCassandra migrated with full consistency
~30 TBHadoop data moved to Dataproc on GCS
4core platforms upgraded in flight
The challenge
What stood in the way
Zero-downtime expectation
A live telecom platform that could not stop.
~90 TB across clouds
Cassandra data moved with 100% consistency, then the delta.
Upgrades during the move
Infrastructure versions upgraded as part of the migration.
Impala to Presto and BigQuery
Not an in-place replacement: syntax and behaviour both change.
Secrets management
All confidential data moved to GKE secret management.
VMs to Kubernetes
Services and infrastructure (RabbitMQ, Hazelcast) re-platformed onto Kubernetes.
Assessment & approach
From diagnosis to design
What we found
- Live ingest through Flume, NiFi, Kafka and RabbitMQ into Storm
- ~90 TB in a Cassandra ring with a Hazelcast in-memory grid
- Self-hosted Cloudera Hadoop (~30 TB) with Impala queries
- Monitoring on Nagios, Munin and Kibana
What we did
- Stateless services moved from VMs to Kubernetes
- Cassandra (~90 TB) and CDH (~30 TB) migrated, then the delta synchronised
- Self-hosted CDH replaced by Dataproc, with data on GCS instead of HDFS
- Impala rewritten for Presto, tuned per query and per node
- Secrets moved into GKE secret management
The solution
How it is built
Platform data flowArchitecture
Sources
Source servers
Log serverFlume agent
Oracle DBvia Sqoop
S3 → GCSobject storage
Ingest & stream
Emitter
NiFiflow management
Kafka · RabbitMQmessage brokers
Stormstream processing
Store & serve
Cassandra ring~90 TB
Hazelcastin-memory grid
Dataproc + PrestoHadoop on GCS
Kong · web servicesMySQL · FTP reports
On Google CloudGKEDataprocBigQueryGCSGKE secretsNagios · Kibana
Upgraded during the moveVersions
RabbitMQ3.2 3.7.15
Hazelcast3.3 3.11
Apache NiFi1.1 1.4
Cloudera CDH5.3 5.15
Technology
Google CloudGKEDataprocBigQueryCassandraKafkaRabbitMQNiFiStormHazelcastPrestoJava 8Kong
Start a conversation
Tell us what you need to build, move or run.
An engineer reads every enquiry, not a sales script. We come back with an approach, a team and a plan you can hold us to.