An approach to archive important and sensitive non-disruptive operations using Akka Streams and Kubernetes

ChatWork migrated our chat messaging system from PHP to Scala and our database from Aurora to HBase at the end of 2016. After the data migration, adjustments were made on the production data and they were not disruptive. We adopted Akka Streams rather than Spark, which was used in the data migration. We achieved non-disruptive adjustment for 2 billion records of production data even though we had experienced big changes from RDBMS to NoSQL and started a new DB operation. In this talk, I will share how the high throughput, flexibility, and extensibility of Akka Streams overcame the challenges above. In addition, I will share how we completed this sensitive operation safely by dynamic throttling control using a combination of Akka Streams and Kubernetes.

Session length

40 minutes

Language of the presentation

Japanese

Target audience

Intermediate: Requires a basic knowledge of the area

Who is your session intended to

People who need non-stop operations on production DB
People who are interested in Akka and modern container orchestration tools

Speaker

awekuit (ChatWork)