Data Engineer
K Stone Corporation Co., Ltd.- Be a strong advocate for a culture of process and data quality across the teams.
- Design, build, administer and scale data processing pipelines (batch/streaming) from structured/semi-structure/unstructured types of data sources (CSV, JSON, Parquet, tables, Kafka, etc.)
- Improve scalability, stability, accuracy, speed and efficiency of current data systems.
- Responsible for design, building, testing and deployment of new libraries, frameworks for core systems with the high standard of quality.
- Work cross-functional teams of functional analyst, project manager, other engineers as well as work closely with the data science and product team in an agile way (or project base in some cases) on requirements and deliverable
- Ensure SLA and provide the greatest experience for users by collaborating closely with the Data Operations team.
- Ensure compliance with data governance and security policies.
- Strong experience in Big data development
- Extensive experience in the automated software and data pipeline development and deployment flow (CI/CD)
- Hands-on experience in data modeling and creating algorithms that turn data into optimal, practical, and valuable information.
- Strong coding ability in these types of programming languages; Apache Spark, Python/PySpark, SQL
- Good understanding of the Data Management concept
- Knowledge of other data and analytics tools, such as key-value data stores, in-memory data stores, real-time analytics databases, etc.
- Knowledge of the Azure cloud computing service
- Experience in working with Azure Databricks
- Experience and know-how in working with banking data
- Knowledge of the Scala programming language
- Staff training and development
- Overtime
- 5-day work week
- Social security
- Health insurance
- Annual trip or party
K.Stone Corporation Ltd. offer world class recruitment and outsourcing services to promote your success.
We give you the ability to leverage of having the right people for the right job.
Our organization aims to deliver exceptional service to both client and candidate.
Contractual and permanent placement of candidates is the exclusive business of our company.