Apache Kafka
 |
|
| Developer(s) | Apache Software Foundation |
|---|---|
| Stable release | 0.10.0 / May 2016 |
| Development status | Active |
| Written in | Scala |
| Operating system | Cross-platform |
| Type | Message broker |
| License | Apache License 2.0 |
| Website | kafka |
Apache Kafka is an open-source message broker project developed by the Apache Software Foundation written in Scala. The project aims to provide a unified, high-throughput, low-latency platform for handling real-time data feeds. It is, in its essence, a "massively scalable pub/sub message queue architected as a distributed transaction log",[2] making it highly valuable for enterprise infrastructures.
The design is heavily influenced by transaction logs.[3]
History
Apache Kafka was originally developed by LinkedIn, and was subsequently open sourced in early 2011. Graduation from the Apache Incubator occurred on 23 October 2012. In November 2014, several engineers who built Kafka at LinkedIn created a new company named Confluent[4] with a focus on Kafka.
Enterprises that use Kafka
The following is a list of notable enterprises that have used or are using Kafka:
Kafka performance
Due to its ability to scale massively and it being largely used by enterprise-level infrastructures, tracking Kafka performance has become an increasingly important issue. There are currently several open-source, like Linkedin's Burrow, as well as paid monitoring platforms, like Datadog.
The key metrics[13] these platforms track consist of:
- Kafka server (broker) metrics
- Producer metrics
- Consumer metrics for both Version 0.8.2.2 and 0.9.0.0+
Kafka is also often used in conjunction with ZooKeeper for deployment management, which necessitates monitoring its metrics alongside Kafka clusters[14].
See also
- Apache ActiveMQ
- Apache Samza
- StormMQ
- Apache Qpid
- Message-oriented middleware
- Enterprise messaging system
- Enterprise Integration Patterns
- Service-oriented architecture
- Event-driven SOA
References
- ↑ Repository Mirror at GitHub
- ↑ Monitoring Kafka performance metrics, Datadog Engineering Blog, accessed 23 May 2016/
- ↑ The Log: What every software engineer should know about real-time data's unifying abstraction, LinkedIn Engineering Blog, accessed 5 May 2014
- ↑ Lua error in package.lua at line 80: module 'strict' not found.
- ↑ Lua error in package.lua at line 80: module 'strict' not found.
- ↑ Lua error in package.lua at line 80: module 'strict' not found.
- ↑ Lua error in package.lua at line 80: module 'strict' not found.
- ↑ Lua error in package.lua at line 80: module 'strict' not found.
- ↑ Lua error in package.lua at line 80: module 'strict' not found.
- ↑ Lua error in package.lua at line 80: module 'strict' not found.
- ↑ Lua error in package.lua at line 80: module 'strict' not found.
- ↑ Lua error in package.lua at line 80: module 'strict' not found.
- ↑ Lua error in package.lua at line 80: module 'strict' not found.
- ↑ Lua error in package.lua at line 80: module 'strict' not found.
External links
- Apache Kafka website
- Discussion of project's design
- Github mirror
- Apache Kafka presentation by Morten Kjetland
- Comparison with RabbitMQ on Quora
- Comparison with RabbitMQ on the Kafka developer mailing list
- Comparison with RabbitMQ and ZeroMQ on Stackoverflow
- Intra-cluster Replication in Apache Kafka
- Kafka Users Mailing List Discussions
- LinkedIn open sourcing announcement
<templatestyles src="Asbox/styles.css"></templatestyles>
 |
This free-software-related article is a stub. You can help Wikipedia by expanding it. |
