[38;2;255;187;0m[4mAwesome[0m[38;2;255;187;0m[4m [0m[38;2;255;187;0m[4mStreaming[0m[38;2;255;187;0m[4m  [0m[38;5;14m[1m[4m![0m[38;2;255;187;0m[4mAwesome[0m[38;5;14m[1m[4m [0m[38;5;14m[1m[4m(https://cdn.rawgit.com/sindresorhus/awesome/d7305f38d29fed78fa85652e3a63e154dd8e8829/media/badge.svg)[0m[38;2;255;187;0m[4m [0m[38;2;255;187;0m[4m(https://github.com/sindresorhus/awesome)[0m[38;2;255;187;0m[4m [0m[38;5;14m[1m[4m![0m[38;2;255;187;0m[4mBuild[0m[38;2;255;187;0m[4m [0m[38;2;255;187;0m[4mStatus[0m[38;5;14m[1m[4m [0m
[38;5;14m[1m[4m(https://github.com/manuzhang/awesome-streaming/workflows/build/badge.svg)[0m[38;2;255;187;0m[4m [0m[38;2;255;187;0m[4m(https://github.com/manuzhang/awesome-streaming/actions)[0m

[38;5;12mA[39m[38;5;12m [39m[38;5;12mcurated[39m[38;5;12m [39m[38;5;12mlist[39m[38;5;12m [39m[38;5;12mof[39m[38;5;12m [39m[38;5;12mawesome[39m[38;5;12m [39m[38;5;14m[1mstreaming[0m[38;5;14m[1m [0m[38;5;14m[1m(stream[0m[38;5;14m[1m [0m[38;5;14m[1mprocessing)[0m[38;5;12m [39m[38;5;12m(http://radar.oreilly.com/2015/08/the-world-beyond-batch-streaming-101.html)[39m[38;5;12m [39m[38;5;12mframeworks,[39m[38;5;12m [39m[38;5;12mapplications,[39m[38;5;12m [39m[38;5;12mreadings[39m[38;5;12m [39m[38;5;12mand[39m[38;5;12m [39m[38;5;12mother[39m[38;5;12m [39m[38;5;12mresources.[39m[38;5;12m [39m[38;5;12mInspired[39m[38;5;12m [39m[38;5;12mby[39m[38;5;12m [39m[38;5;14m[1mother[0m[38;5;14m[1m [0m[38;5;14m[1mawesome[0m[38;5;14m[1m [0m[38;5;14m[1mprojects[0m[38;5;12m [39m
[38;5;12m(https://github.com/sindresorhus/awesome).[39m[38;5;12m [39m

[38;2;255;187;0m[4mWebsite[0m

[38;5;14m[1mhttps://manuzhang.github.io/awesome-streaming/[0m[38;5;12m (https://manuzhang.github.io/awesome-streaming/) is a more dynamic website where you can find [39m[38;5;14m[1mupdates[0m[38;5;12m of the awesome projects here.[39m

[38;2;255;187;0m[4mTable of Contents[0m

[38;5;12m- [39m[38;5;14m[1mStreaming Engine[0m[38;5;12m (#streaming-engine)[39m
[38;5;12m- [39m[38;5;14m[1mStreaming Library[0m[38;5;12m (#streaming-library)[39m
[38;5;12m- [39m[38;5;14m[1mStreaming Application[0m[38;5;12m (#streaming-application)[39m
[38;5;12m- [39m[38;5;14m[1mIoT[0m[38;5;12m (#iot)[39m
[38;5;12m- [39m[38;5;14m[1mDSL[0m[38;5;12m (#dsl)[39m
[38;5;12m- [39m[38;5;14m[1mData Pipeline[0m[38;5;12m (#data-pipeline)[39m
[38;5;12m- [39m[38;5;14m[1mOnline Machine Learning[0m[38;5;12m (#online-machine-learning)[39m
[38;5;12m- [39m[38;5;14m[1mStreaming SQL[0m[38;5;12m (#streaming-sql)[39m
[38;5;12m- [39m[38;5;14m[1mToolkit[0m[38;5;12m (#toolkit)[39m
[38;5;12m- [39m[38;5;14m[1mBenchmark[0m[38;5;12m (#benchmark)[39m
[38;5;12m- [39m[38;5;14m[1mClosed Source[0m[38;5;12m (#closed-source)[39m
[38;5;12m- [39m[38;5;14m[1mReadings[0m[38;5;12m (#readings)[39m

[38;2;255;187;0m[4mStreaming Engine[0m

[38;5;12m- [39m[38;5;14m[1mApache Apex[0m[38;5;12m (https://github.com/apache/apex-core) [39m[38;5;14m[1mJava[0m[38;5;12m  - unified platform for big data stream and batch processing.[39m
[38;5;12m- [39m[38;5;14m[1mApache Ballista[0m[38;5;12m (https://github.com/apache/arrow-ballista) [39m[38;5;14m[1mRust[0m[38;5;12m  - distributed compute platform powered by Apache Arrow.[39m
[38;5;12m- [39m[38;5;14m[1mApache Flink[0m[38;5;12m (https://github.com/apache/flink) [39m[38;5;14m[1mJava[0m[38;5;12m  - system for high-throughput, low-latency data stream processing that supports stateful computation, data-driven windowing semantics and iterative stream processing.[39m
[38;5;12m- [39m[38;5;14m[1mApache Heron (incubating)[0m[38;5;12m (https://github.com/apache/incubator-heron) [39m[38;5;14m[1mJava[0m[38;5;12m  - a realtime, distributed, fault-tolerant stream processing engine from Twitter.[39m
[38;5;12m- [39m[38;5;14m[1mApache Samza[0m[38;5;12m (https://github.com/apache/samza) [39m[38;5;14m[1mScala/Java[0m[38;5;12m  - distributed stream processing framework that build on Kafka(messaging, storage) and YARN(fault tolerance, processor isolation, security and resource management).[39m
[38;5;12m- [39m[38;5;14m[1mApache Spark Streaming[0m[38;5;12m (https://github.com/apache/spark) [39m[38;5;14m[1mScala[0m[38;5;12m  - makes it easy to build scalable fault-tolerant streaming applications.[39m
[38;5;12m- [39m[38;5;14m[1mApache Storm[0m[38;5;12m (https://github.com/apache/storm) [39m[38;5;14m[1mClojure/Java[0m[38;5;12m  - distributed real-time computation system. Storm is to stream processing what Hadoop is to batch processing. [39m
[38;5;12m- [39m[38;5;14m[1mAthenaX[0m[38;5;12m (https://github.com/uber/AthenaX) [39m[38;5;14m[1mJava[0m[38;5;12m  - Uber's Stream Analytics Framework used in production[39m
[38;5;12m- [39m[38;5;14m[1mBytewax[0m[38;5;12m (https://github.com/bytewax/bytewax) [39m[38;5;14m[1mPython[0m[38;5;12m  - data parallel, distributed, stateful stream processing framework.[39m
[38;5;12m- [39m[38;5;14m[1mFaust[0m[38;5;12m (https://github.com/robinhood/faust) [39m[38;5;14m[1mPython[0m[38;5;12m  - stream processing library, porting the ideas from Kafka Streams to Python[39m
[38;5;12m- [39m[38;5;14m[1mGearpump[0m[38;5;12m (https://github.com/gearpump/gearpump) [39m[38;5;14m[1mScala[0m[38;5;12m  - lightweight real-time distributed streaming engine built on Akka.[39m
[38;5;12m- [39m[38;5;14m[1mHazelcast Jet[0m[38;5;12m (https://github.com/hazelcast/hazelcast-jet) [39m[38;5;14m[1mJava[0m[38;5;12m  - A general purpose distributed data processing engine, built on top of Hazelcast.[39m
[38;5;12m- [39m[38;5;14m[1mhailstorm[0m[38;5;12m (https://github.com/hailstorm-hs/hailstorm) [39m[38;5;14m[1mHaskell[0m[38;5;12m  - distributed stream processing with exactly-once semantics based on Storm.[39m
[38;5;12m- [39m[38;5;14m[1mMaki Nage[0m[38;5;12m (https://github.com/maki-nage/makinage) [39m[38;5;14m[1mPython[0m[38;5;12m  - A stream processing framework for data scientists, based on Kafka and ReactiveX.[39m
[38;5;12m- [39m[38;5;14m[1mmantis[0m[38;5;12m (https://github.com/Netflix/mantis) [39m[38;5;14m[1mJava[0m[38;5;12m  - Netflix's platform to build an ecosystem of realtime stream processing applications[39m
[38;5;12m- [39m[38;5;14m[1mmupd8(muppet)[0m[38;5;12m (https://github.com/walmartlabs/mupd8) [39m[38;5;14m[1mScala/Java[0m[38;5;12m  - mapReduce-style framework for processing fast/streaming data.[39m
[38;5;12m- [39m[38;5;14m[1mNumaflow[0m[38;5;12m (https://github.com/numaproj/numaflow) [39m[38;5;14m[1mJava/Python/Go/Rust[0m[38;5;12m  - Kubernetes native stream processing platform with language agnostic framework. Scalable and cost-efficient[39m
[38;5;12m- [39m[38;5;14m[1mOnyx[0m[38;5;12m (https://github.com/onyx-platform/onyx) [39m[38;5;14m[1mClojure[0m[38;5;12m  - Distributed, masterless, high performance, fault tolerant data processing.[39m
[38;5;12m- [39m[38;5;14m[1mPathway[0m[38;5;12m (https://github.com/pathwaycom/pathway) [39m[38;5;14m[1mPython[0m[38;5;12m  - The fastest data processing engine supporting unified workflows for batch, streaming data, and LLM applications.[39m
[38;5;12m- [39m[38;5;14m[1ms4[0m[38;5;12m (https://github.com/apache/incubator-s4) [39m[38;5;14m[1mJava[0m[38;5;12m  - general-purpose, distributed, scalable, fault-tolerant, pluggable platform that allows programmers to easily develop applications for processing continuous unbounded streams of data.[39m
[38;5;12m- [39m[38;5;14m[1mSABER[0m[38;5;12m (https://github.com/lsds/Saber) [39m[38;5;14m[1mJava/C[0m[38;5;12m  - Window-Based Hybrid CPU/GPU Stream Processing Engine.[39m
[38;5;12m- [39m[38;5;14m[1mScramjet Cloud Platform[0m[38;5;12m (https://github.com/scramjetorg/transform-hub) [39m[38;5;14m[1mPython/JavaScript/Node.js[0m[38;5;12m  - data processing engine for running multiple data processing apps (sequences) written in Python, JavaScript or TypeScript [39m
[38;5;12m- [39m[38;5;14m[1mSPQR[0m[38;5;12m (https://github.com/ottogroup/SPQR) [39m[38;5;14m[1mJava[0m[38;5;12m  - dynamic framework for processing high volumn data streams through pipelines.[39m
[38;5;12m- [39m[38;5;14m[1mtigon[0m[38;5;12m (https://github.com/caskdata/tigon) [39m[38;5;14m[1mC++/Java[0m[38;5;12m  - high throughput real-time streaming processing framework built on Hadoop and HBase.[39m
[38;5;12m- [39m[38;5;14m[1mTeknek[0m[38;5;12m (https://github.com/edwardcapriolo/teknek-core) [39m[38;5;14m[1mJava[0m[38;5;12m  - Simple elegant stream processing with interactive prototying shell SOL (Stream Operator Language)[39m
[38;5;12mMesos, designed for high performance data processing jobs that require flexibility & control.[39m
[38;5;12m- [39m[38;5;14m[1mTrill[0m[38;5;12m (https://github.com/Microsoft/trill) [39m[38;5;14m[1m.NET/C#[0m[38;5;12m  - Trill is a high-performance one-pass in-memory streaming analytics engine from Microsoft Research.[39m
[38;5;12m-[39m[38;5;12m [39m[38;5;14m[1mWallaroo[0m[38;5;12m [39m[38;5;12m(https://github.com/WallarooLabs/wallaroo)[39m[38;5;12m [39m[38;5;14m[1mPython[0m[38;5;12m  [39m[38;5;12m-[39m[38;5;12m [39m[38;5;12mA[39m[38;5;12m [39m[38;5;12mfast,[39m[38;5;12m [39m[38;5;12mstream-processing[39m[38;5;12m [39m[38;5;12mframework.[39m[38;5;12m [39m[38;5;12mWallaroo[39m[38;5;12m [39m[38;5;12mmakes[39m[38;5;12m [39m[38;5;12mit[39m[38;5;12m [39m[38;5;12measy[39m[38;5;12m [39m[38;5;12mto[39m[38;5;12m [39m[38;5;12mreact[39m[38;5;12m [39m[38;5;12mto[39m[38;5;12m [39m[38;5;12mdata[39m[38;5;12m [39m[38;5;12min[39m[38;5;12m [39m[38;5;12mreal-time.[39m[38;5;12m [39m[38;5;12mBy[39m[38;5;12m [39m[38;5;12meliminating[39m[38;5;12m [39m[38;5;12minfrastructure[39m[38;5;12m [39m[38;5;12mcomplexity,[39m[38;5;12m [39m[38;5;12mgoing[39m[38;5;12m [39m[38;5;12mfrom[39m[38;5;12m [39m[38;5;12mprototype[39m[38;5;12m [39m[38;5;12mto[39m[38;5;12m [39m[38;5;12mproduction[39m[38;5;12m [39m[38;5;12mhas[39m[38;5;12m [39m
[38;5;12mnever[39m[38;5;12m [39m[38;5;12mbeen[39m[38;5;12m [39m[38;5;12msimpler.[39m
[38;5;12m- [39m[38;5;14m[1mLightSaber[0m[38;5;12m (https://github.com/lsds/LightSaber) [39m[38;5;14m[1mC++[0m[38;5;12m  - Multi-core Window-Based Stream Processing Engine. LightSaber uses code generation for efficient window aggregation.[39m
[38;5;12m- [39m[38;5;14m[1mHStreamDB[0m[38;5;12m (https://github.com/hstreamdb/hstream) [39m[38;5;14m[1mHaskell[0m[38;5;12m  - The streaming database built for IoT data storage and real-time processing.[39m
[38;5;12m- [39m[38;5;14m[1mKuiper[0m[38;5;12m (https://github.com/emqx/kuiper) [39m[38;5;14m[1mGolang[0m[38;5;12m  - An edge lightweight IoT data analytics/streaming software implemented by Golang, and it can be run at all kinds of resource-constrained edge devices.[39m
[38;5;12m- [39m[38;5;14m[1mWindFlow[0m[38;5;12m (https://paragroup.github.io/WindFlow) [39m[38;5;14m[1mC++[0m[38;5;12m  - A C++17 Data Stream Processing Parallel Library for Multicores and GPUs[39m

[38;2;255;187;0m[4mStreaming Library[0m

[38;5;12m- [39m[38;5;14m[1mApache Kafka Streams[0m[38;5;12m (https://github.com/apache/kafka) [39m[38;5;14m[1mJava[0m[38;5;12m  - lightweight stream processing library included in Apache Kafka (since 0.10 version).[39m
[38;5;12m- [39m[38;5;14m[1mStreamiz[0m[38;5;12m (https://github.com/LGouellec/kafka-streams-dotnet) [39m[38;5;14m[1mC#[0m[38;5;12m  - a .Net Stream Processing Library for Apache Kafka[39m
[38;5;12m- [39m[38;5;14m[1mAkka Streams[0m[38;5;12m (https://github.com/akka/akka) [39m[38;5;14m[1mScala[0m[38;5;12m  - stream processing library on Akka Actors.[39m
[38;5;12m- [39m[38;5;14m[1mDaggy[0m[38;5;12m (https://github.com/synacker/daggy) [39m[38;5;14m[1mC++[0m[38;5;12m  - real-time streams aggregation and catching. [39m
[38;5;12m- [39m[38;5;14m[1mBenthos[0m[38;5;12m (https://github.com/Jeffail/benthos) [39m[38;5;14m[1mGo[0m[38;5;12m  - Benthos is a high performance and resilient message streaming service, able to connect various sources and sinks and perform arbitrary actions, transformations and filters on payloads[39m
[38;5;12m- [39m[38;5;14m[1mFS2(prev. 'Scalaz-Stream')[0m[38;5;12m (https://github.com/functional-streams-for-scala/fs2) [39m[38;5;14m[1mScala[0m[38;5;12m  - Compositional, streaming I/O library for Scala.[39m
[38;5;12m-[39m[38;5;12m [39m[38;5;14m[1mFastStream[0m[38;5;12m [39m[38;5;12m(https://github.com/airtai/faststream)[39m[38;5;12m [39m[38;5;14m[1mPython[0m[38;5;12m  [39m[38;5;12m-[39m[38;5;12m [39m[38;5;12mpowerful[39m[38;5;12m [39m[38;5;12mand[39m[38;5;12m [39m[38;5;12measy-to-use[39m[38;5;12m [39m[38;5;12mPython[39m[38;5;12m [39m[38;5;12mlibrary[39m[38;5;12m [39m[38;5;12msimplifying[39m[38;5;12m [39m[38;5;12mthe[39m[38;5;12m [39m[38;5;12mprocess[39m[38;5;12m [39m[38;5;12mof[39m[38;5;12m [39m[38;5;12mwriting[39m[38;5;12m [39m[38;5;12mproducers[39m[38;5;12m [39m[38;5;12mand[39m[38;5;12m [39m[38;5;12mconsumers[39m[38;5;12m [39m[38;5;12mfor[39m[38;5;12m [39m[38;5;12mmessage[39m[38;5;12m [39m[38;5;12mqueues,[39m[38;5;12m [39m[38;5;12mhandling[39m[38;5;12m [39m[38;5;12mall[39m[38;5;12m [39m[38;5;12mthe[39m[38;5;12m [39m[38;5;12mparsing,[39m[38;5;12m [39m[38;5;12mnetworking[39m[38;5;12m [39m[38;5;12mand[39m[38;5;12m [39m[38;5;12mdocumentation[39m[38;5;12m [39m
[38;5;12mgeneration[39m[38;5;12m [39m[38;5;12mautomatically.[39m[38;5;12m [39m[38;5;12mSupports[39m[38;5;12m [39m[38;5;12mmultiple[39m[38;5;12m [39m[38;5;12mprotocols[39m[38;5;12m [39m[38;5;12msuch[39m[38;5;12m [39m[38;5;12mas[39m[38;5;12m [39m[38;5;12mApache[39m[38;5;12m [39m[38;5;12mKafka,[39m[38;5;12m [39m[38;5;12mRabbitMQ[39m[38;5;12m [39m[38;5;12mand[39m[38;5;12m [39m[38;5;12malike.[39m
[38;5;12m- [39m[38;5;14m[1mmonix[0m[38;5;12m (https://github.com/monix/monix) [39m[38;5;14m[1mScala[0m[38;5;12m  - high-performance Scala / Scala.js library for composing asynchronous and event-based programs.[39m
[38;5;12m-[39m[38;5;12m [39m[38;5;14m[1mQuix[0m[38;5;14m[1m [0m[38;5;14m[1mStreams[0m[38;5;12m [39m[38;5;12m(https://github.com/quixio/quix-streams)[39m[38;5;12m [39m[38;5;14m[1mPython[0m[38;5;12m  [39m[38;5;12m-[39m[38;5;12m [39m[38;5;12ma[39m[38;5;12m [39m[38;5;12mstreaming[39m[38;5;12m [39m[38;5;12mlibrary[39m[38;5;12m [39m[38;5;12moriginally[39m[38;5;12m [39m[38;5;12mdesigned[39m[38;5;12m [39m[38;5;12mfor[39m[38;5;12m [39m[38;5;12mthe[39m[38;5;12m [39m[38;5;12mMcLaren[39m[38;5;12m [39m[38;5;12mFormula[39m[38;5;12m [39m[38;5;12m1[39m[38;5;12m [39m[38;5;12mracing[39m[38;5;12m [39m[38;5;12mteam[39m[38;5;12m [39m[38;5;12mthat[39m[38;5;12m [39m[38;5;12mcan[39m[38;5;12m [39m[38;5;12mprocess[39m[38;5;12m [39m[38;5;12mhigh[39m[38;5;12m [39m[38;5;12mvolumes[39m[38;5;12m [39m[38;5;12mof[39m[38;5;12m [39m[38;5;12mtime-series[39m[38;5;12m [39m[38;5;12mdata[39m[38;5;12m [39m[38;5;12mwith[39m[38;5;12m [39m[38;5;12mup[39m[38;5;12m [39m[38;5;12mto[39m[38;5;12m [39m[38;5;12mnanosecond[39m[38;5;12m [39m[38;5;12mprecision[39m[38;5;12m [39m[38;5;12musing[39m[38;5;12m [39m
[38;5;12mApache[39m[38;5;12m [39m[38;5;12mKafka[39m[38;5;12m [39m[38;5;12mas[39m[38;5;12m [39m[38;5;12ma[39m[38;5;12m [39m[38;5;12mmessage[39m[38;5;12m [39m[38;5;12mbroker.[39m
[38;5;12m-[39m[38;5;12m [39m[38;5;14m[1mScramjet[0m[38;5;14m[1m [0m[38;5;14m[1mNode.js[0m[38;5;12m [39m[38;5;12m(https://github.com/scramjetorg/framework-js)[39m[38;5;12m [39m[38;5;12m-[39m[38;5;12m [39m[38;5;14m[1mNode.js[0m[38;5;12m  [39m[38;5;12mfunctional[39m[38;5;12m [39m[38;5;12mreactive[39m[38;5;12m [39m[38;5;12mstream[39m[38;5;12m [39m[38;5;12mprogramming[39m[38;5;12m [39m[38;5;12mframework[39m[38;5;12m [39m[38;5;12mwritten[39m[38;5;12m [39m[38;5;12mon[39m[38;5;12m [39m[38;5;12mtop[39m[38;5;12m [39m[38;5;12mof[39m[38;5;12m [39m[38;5;12mNode.js[39m[38;5;12m [39m[38;5;12mobject[39m[38;5;12m [39m[38;5;12mstreams[39m[38;5;12m [39m[38;5;12m+[39m[38;5;12m [39m[38;5;14m[1mthe[0m[38;5;14m[1m [0m[38;5;14m[1mlegacy[0m[38;5;14m[1m [0m[38;5;14m[1mScramjet.js[0m[38;5;14m[1m [0m[38;5;14m[1mversion[0m[38;5;12m [39m
[38;5;12m(https://github.com/scramjetorg/scramjet)[39m
[38;5;12m- [39m[38;5;14m[1mScramjet Python[0m[38;5;12m (https://github.com/scramjetorg/framework-python) - [39m[38;5;14m[1mPython[0m[38;5;12m  functional reactive stream programming framework written from scratch operating on object, string and buffer streams.[39m
[38;5;12m- [39m[38;5;14m[1mScramjet C++[0m[38;5;12m (https://github.com/scramjetorg/framework-cpp) - [39m[38;5;14m[1mC++[0m[38;5;12m  functional reactive stream programming framework written on top of Node.js object streams.[39m
[38;5;12m-[39m[38;5;12m [39m[38;5;14m[1mStreamline[0m[38;5;12m [39m[38;5;12m(https://github.com/hortonworks/streamline)[39m[38;5;12m [39m[38;5;14m[1mJava[0m[38;5;12m  [39m[38;5;12m-[39m[38;5;12m [39m[38;5;12mStream[39m[38;5;12m [39m[38;5;12mAnalytics[39m[38;5;12m [39m[38;5;12mFramework[39m[38;5;12m [39m[38;5;12mby[39m[38;5;12m [39m[38;5;12mHortonworks,[39m[38;5;12m [39m[38;5;12mdesigned[39m[38;5;12m [39m[38;5;12mas[39m[38;5;12m [39m[38;5;12ma[39m[38;5;12m [39m[38;5;12mwrapper[39m[38;5;12m [39m[38;5;12maround[39m[38;5;12m [39m[38;5;12mexisting[39m[38;5;12m [39m[38;5;12mstreaming[39m[38;5;12m [39m[38;5;12msolutions[39m[38;5;12m [39m[38;5;12mlike[39m[38;5;12m [39m[38;5;12mStorm.[39m[38;5;12m [39m[38;5;12mAimed[39m[38;5;12m [39m[38;5;12mto[39m[38;5;12m [39m[38;5;12mallow[39m[38;5;12m [39m[38;5;12musers[39m[38;5;12m [39m[38;5;12mto[39m[38;5;12m [39m[38;5;12mdrag-and-drop[39m[38;5;12m [39m[38;5;12mstreaming[39m[38;5;12m [39m[38;5;12mcomponents[39m
[38;5;12mto[39m[38;5;12m [39m[38;5;12mfocus[39m[38;5;12m [39m[38;5;12mon[39m[38;5;12m [39m[38;5;12mbusiness[39m[38;5;12m [39m[38;5;12mlogic.[39m
[38;5;12m- [39m[38;5;14m[1mStreamAlert[0m[38;5;12m (https://github.com/airbnb/streamalert) [39m[38;5;14m[1mPython[0m[38;5;12m  - Airbnb's Real-time Data Analysis and Alerting.[39m
[38;5;12m- [39m[38;5;14m[1mSwave[0m[38;5;12m (https://github.com/sirthias/swave) [39m[38;5;14m[1mScala[0m[38;5;12m  - A lightweight Reactive Streams Infrastructure Toolkit for Scala.[39m
[38;5;12m-[39m[38;5;12m [39m[38;5;14m[1mStreamz[0m[38;5;12m [39m[38;5;12m(https://github.com/python-streamz/streamz)[39m[38;5;12m [39m[38;5;14m[1mPython[0m[38;5;12m  [39m[38;5;12m-[39m[38;5;12m [39m[38;5;12mA[39m[38;5;12m [39m[38;5;12mlightweight[39m[38;5;12m [39m[38;5;12mlibrary[39m[38;5;12m [39m[38;5;12mfor[39m[38;5;12m [39m[38;5;12mbuilding[39m[38;5;12m [39m[38;5;12mpipelines[39m[38;5;12m [39m[38;5;12mto[39m[38;5;12m [39m[38;5;12mmanage[39m[38;5;12m [39m[38;5;12mcontinuous[39m[38;5;12m [39m[38;5;12mstreams[39m[38;5;12m [39m[38;5;12mof[39m[38;5;12m [39m[38;5;12mdata;[39m[38;5;12m [39m[38;5;12msupports[39m[38;5;12m [39m[38;5;12mcomplex[39m[38;5;12m [39m[38;5;12mpipelines[39m[38;5;12m [39m[38;5;12mthat[39m[38;5;12m [39m[38;5;12minvolve[39m[38;5;12m [39m[38;5;12mbranching,[39m[38;5;12m [39m[38;5;12mjoining,[39m[38;5;12m [39m[38;5;12mflow[39m[38;5;12m [39m[38;5;12mcontrol,[39m[38;5;12m [39m[38;5;12mfeedback,[39m[38;5;12m [39m[38;5;12mback[39m[38;5;12m [39m
[38;5;12mpressure,[39m[38;5;12m [39m[38;5;12mand[39m[38;5;12m [39m[38;5;12mso[39m[38;5;12m [39m[38;5;12mon.[39m
[38;5;12m- [39m[38;5;14m[1mStream Ops[0m[38;5;12m (https://github.com/nanosai/stream-ops-java) [39m[38;5;14m[1mJava[0m[38;5;12m  - A fully embeddable data streaming engine and stream processing API for Java.[39m
[38;5;12m- [39m[38;5;14m[1mSubstation[0m[38;5;12m (https://github.com/brexhq/substation) [39m[38;5;14m[1mGo[0m[38;5;12m  - Substation is a cloud native data pipeline and transformation toolkit written in Go.[39m
[38;5;12m-[39m[38;5;12m [39m[38;5;14m[1mTributary[0m[38;5;12m [39m[38;5;12m(https://github.com/timkpaine/tributary)[39m[38;5;12m [39m[38;5;14m[1mPython[0m[38;5;12m  [39m[38;5;12m-[39m[38;5;12m [39m[38;5;12mA[39m[38;5;12m [39m[38;5;12mpython[39m[38;5;12m [39m[38;5;12mlibrary[39m[38;5;12m [39m[38;5;12mfor[39m[38;5;12m [39m[38;5;12mconstructing[39m[38;5;12m [39m[38;5;12mdataflow[39m[38;5;12m [39m[38;5;12mgraphs.[39m[38;5;12m [39m[38;5;12mSupports[39m[38;5;12m [39m[38;5;12msynchronous,[39m[38;5;12m [39m[38;5;12mreactive[39m[38;5;12m [39m[38;5;12mdata[39m[38;5;12m [39m[38;5;12mstreams[39m[38;5;12m [39m[38;5;12mbuilt[39m[38;5;12m [39m[38;5;12musing[39m[38;5;12m [39m[38;5;12mpython[39m[38;5;12m [39m[38;5;12mgenerators[39m[38;5;12m [39m[38;5;12mthat[39m[38;5;12m [39m[38;5;12mmimic[39m[38;5;12m [39m[38;5;12mcomplex[39m[38;5;12m [39m[38;5;12mevent[39m[38;5;12m [39m[38;5;12mprocessors,[39m[38;5;12m [39m[38;5;12mas[39m[38;5;12m [39m[38;5;12mwell[39m[38;5;12m [39m[38;5;12mas[39m[38;5;12m [39m
[38;5;12mlazily-evaluated[39m[38;5;12m [39m[38;5;12macyclic[39m[38;5;12m [39m[38;5;12mgraphs[39m[38;5;12m [39m[38;5;12mand[39m[38;5;12m [39m[38;5;12mfunctional[39m[38;5;12m [39m[38;5;12mcurrying[39m[38;5;12m [39m[38;5;12mstreams.[39m
[38;5;12m-[39m[38;5;12m [39m[38;5;14m[1mYoMo[0m[38;5;12m [39m[38;5;12m(https://github.com/yomorun/yomo)[39m[38;5;12m [39m[38;5;14m[1mGo[0m[38;5;12m  [39m[38;5;12m-[39m[38;5;12m [39m[38;5;12mAn[39m[38;5;12m [39m[38;5;12mopen[39m[38;5;12m [39m[38;5;12msource[39m[38;5;12m [39m[38;5;12mStreaming[39m[38;5;12m [39m[38;5;12mServerless[39m[38;5;12m [39m[38;5;12mFramework[39m[38;5;12m [39m[38;5;12mfor[39m[38;5;12m [39m[38;5;12mbuilding[39m[38;5;12m [39m[38;5;12mLow-latency[39m[38;5;12m [39m[38;5;12mGeo-distributed[39m[38;5;12m [39m[38;5;12msystem.[39m[38;5;12m [39m[38;5;12mYoMo[39m[38;5;12m [39m[38;5;12mBuilt[39m[38;5;12m [39m[38;5;12matop[39m[38;5;12m [39m[38;5;14m[1mQUIC[0m[38;5;14m[1m [0m[38;5;14m[1mTransport[0m[38;5;14m[1m [0m[38;5;14m[1mProtocol[0m[38;5;12m [39m[38;5;12m(https://en.wikipedia.org/wiki/QUIC)[39m[38;5;12m [39m[38;5;12mand[39m[38;5;12m [39m[38;5;12mFunctional[39m[38;5;12m [39m
[38;5;12mReactive[39m[38;5;12m [39m[38;5;12mProgramming[39m[38;5;12m [39m[38;5;12minterface.[39m[38;5;12m [39m
[38;5;12m- [39m[38;5;14m[1mMediapipe[0m[38;5;12m (https://github.com/google/mediapipe) - Cross-platform, customizable ML solutions for live and streaming media.[39m

[38;2;255;187;0m[4mStreaming Application[0m

[38;5;12m- [39m[38;5;14m[1mjavactrl-kafka[0m[38;5;12m (https://github.com/javactrl/javactrl-kafka) [39m[38;5;14m[1mJava[0m[38;5;12m  - An application of a stateful stream processing for workflow as Java code (microservices orchestration, business process automation, and more).[39m
[38;5;12m- [39m[38;5;14m[1mstraw[0m[38;5;12m (https://github.com/rwalk/straw) [39m[38;5;14m[1mPython/Java[0m[38;5;12m  - A platform for real-time streaming search.[39m
[38;5;12m- [39m[38;5;14m[1mstorm-crawler[0m[38;5;12m (https://github.com/DigitalPebble/storm-crawler) [39m[38;5;14m[1mJava[0m[38;5;12m  - Web crawler SDK based on Apache Storm.[39m
[38;5;12m- [39m[38;5;14m[1mZilla[0m[38;5;12m (https://github.com/aklivity/zilla) [39m[38;5;14m[1mJava[0m[38;5;12m  - Cross-platform, API gateway built for event-driven architectures and streaming that supports standard protocols such as HTTP, SSE, gRPC, MQTT and the native Kafka protocol.[39m

[38;2;255;187;0m[4mIoT[0m

[38;5;12m- [39m[38;5;14m[1msensorbee[0m[38;5;12m (https://github.com/sensorbee/sensorbee) [39m[38;5;14m[1mGo[0m[38;5;12m  - lightweight stream processing engine for IoT.[39m
[38;5;12m-[39m[38;5;12m [39m[38;5;14m[1mApache[0m[38;5;14m[1m [0m[38;5;14m[1mEdgent[0m[38;5;12m [39m[38;5;12m(https://github.com/apache/incubator-edgent)[39m[38;5;12m [39m[38;5;14m[1mJava[0m[38;5;12m  [39m[38;5;12m-[39m[38;5;12m [39m[38;5;12ma[39m[38;5;12m [39m[38;5;12mprogramming[39m[38;5;12m [39m[38;5;12mmodel[39m[38;5;12m [39m[38;5;12mand[39m[38;5;12m [39m[38;5;12mruntime[39m[38;5;12m [39m[38;5;12mthat[39m[38;5;12m [39m[38;5;12menables[39m[38;5;12m [39m[38;5;12mcontinuous[39m[38;5;12m [39m[38;5;12mstreaming[39m[38;5;12m [39m[38;5;12manalytics[39m[38;5;12m [39m[38;5;12mon[39m[38;5;12m [39m[38;5;12mgateways[39m[38;5;12m [39m[38;5;12mand[39m[38;5;12m [39m[38;5;12medge[39m[38;5;12m [39m[38;5;12mdevices[39m[38;5;12m [39m[38;5;12mwhich[39m[38;5;12m [39m[38;5;12mcan[39m[38;5;12m [39m[38;5;12mwork[39m[38;5;12m [39m[38;5;12mwith[39m[38;5;12m [39m[38;5;12mcentralized[39m[38;5;12m [39m[38;5;12msystems[39m[38;5;12m [39m[38;5;12mto[39m[38;5;12m [39m[38;5;12mprovide[39m[38;5;12m [39m[38;5;12mefficient[39m[38;5;12m [39m
[38;5;12mand[39m[38;5;12m [39m[38;5;12mtimely[39m[38;5;12m [39m[38;5;12manalytics[39m[38;5;12m [39m[38;5;12macross[39m[38;5;12m [39m[38;5;12mthe[39m[38;5;12m [39m[38;5;12mwhole[39m[38;5;12m [39m[38;5;12mIoT[39m[38;5;12m [39m[38;5;12mecosystem:[39m[38;5;12m [39m[38;5;12mfrom[39m[38;5;12m [39m[38;5;12mthe[39m[38;5;12m [39m[38;5;12mcenter[39m[38;5;12m [39m[38;5;12mto[39m[38;5;12m [39m[38;5;12mthe[39m[38;5;12m [39m[38;5;12medge,[39m[38;5;12m [39m[38;5;12mopens[39m[38;5;12m [39m[38;5;12msourced[39m[38;5;12m [39m[38;5;12mby[39m[38;5;12m [39m[38;5;12mIBM.[39m
[38;5;12m- [39m[38;5;14m[1mApache StreamPipes[0m[38;5;12m (https://github.com/apache/incubator-streampipes) [39m[38;5;14m[1mJava[0m[38;5;12m  - a self-service (Industrial) IoT toolbox to enable non-technical users to connect, analyze and explore IoT data streams.[39m

[38;2;255;187;0m[4mDSL[0m

[38;5;12m-[39m[38;5;12m [39m[38;5;14m[1mApache[0m[38;5;14m[1m [0m[38;5;14m[1mBeam[0m[38;5;12m [39m[38;5;12m(https://github.com/apache/beam)[39m[38;5;12m [39m[38;5;14m[1mJava,[0m[38;5;14m[1m [0m[38;5;14m[1mPython,[0m[38;5;14m[1m [0m[38;5;14m[1mSQL,[0m[38;5;14m[1m [0m[38;5;14m[1mScala,[0m[38;5;14m[1m [0m[38;5;14m[1mGo[0m[38;5;12m  [39m[38;5;12m-[39m[38;5;12m [39m[38;5;12munified[39m[38;5;12m [39m[38;5;12mmodel[39m[38;5;12m [39m[38;5;12mand[39m[38;5;12m [39m[38;5;12mset[39m[38;5;12m [39m[38;5;12mof[39m[38;5;12m [39m[38;5;12mlanguage-specific[39m[38;5;12m [39m[38;5;12mSDKs[39m[38;5;12m [39m[38;5;12mfor[39m[38;5;12m [39m[38;5;12mdefining[39m[38;5;12m [39m[38;5;12mand[39m[38;5;12m [39m[38;5;12mexecuting[39m[38;5;12m [39m[38;5;12mdata[39m[38;5;12m [39m[38;5;12mprocessing[39m[38;5;12m [39m[38;5;12mworkflows,[39m[38;5;12m [39m[38;5;12mand[39m[38;5;12m [39m[38;5;12malso[39m[38;5;12m [39m[38;5;12mdata[39m[38;5;12m [39m[38;5;12mingestion[39m[38;5;12m [39m[38;5;12mand[39m[38;5;12m [39m[38;5;12mintegration[39m[38;5;12m [39m[38;5;12mflows,[39m[38;5;12m [39m
[38;5;12msupporting[39m[38;5;12m [39m[38;5;12mEnterprise[39m[38;5;12m [39m[38;5;12mIntegration[39m[38;5;12m [39m[38;5;12mPatterns[39m[38;5;12m [39m[38;5;12m(EIPs)[39m[38;5;12m [39m[38;5;12mand[39m[38;5;12m [39m[38;5;12mDomain[39m[38;5;12m [39m[38;5;12mSpecific[39m[38;5;12m [39m[38;5;12mLanguages[39m[38;5;12m [39m[38;5;12m(DSLs),[39m[38;5;12m [39m[38;5;12mopen[39m[38;5;12m [39m[38;5;12msourced[39m[38;5;12m [39m[38;5;12mby[39m[38;5;12m [39m[38;5;12mGoogle.[39m
[38;5;12m- [39m[38;5;14m[1mcoast[0m[38;5;12m (https://github.com/bkirwi/coast) [39m[38;5;14m[1mScala[0m[38;5;12m  - a DSL that builds DAGs on top of Samza and provides exactly-once semantics.[39m
[38;5;12m- [39m[38;5;14m[1mEsper[0m[38;5;12m (https://github.com/espertechinc/esper) [39m[38;5;14m[1mJava[0m[38;5;12m  - component for complex event processing (CEP) and event series analysis.[39m
[38;5;12m- [39m[38;5;14m[1mStreamparse[0m[38;5;12m (https://github.com/Parsely/streamparse) [39m[38;5;14m[1mPython[0m[38;5;12m  - lets you run Python code against real-time streams of data via Apache Storm.[39m
[38;5;12m-[39m[38;5;12m [39m[38;5;14m[1msummingbird[0m[38;5;12m [39m[38;5;12m(https://github.com/twitter/summingbird)[39m[38;5;12m [39m[38;5;14m[1mScala[0m[38;5;12m  [39m[38;5;12m-[39m[38;5;12m [39m[38;5;12mlibrary[39m[38;5;12m [39m[38;5;12mthat[39m[38;5;12m [39m[38;5;12mlets[39m[38;5;12m [39m[38;5;12myou[39m[38;5;12m [39m[38;5;12mwrite[39m[38;5;12m [39m[38;5;12mMapReduce[39m[38;5;12m [39m[38;5;12mprograms[39m[38;5;12m [39m[38;5;12mthat[39m[38;5;12m [39m[38;5;12mlook[39m[38;5;12m [39m[38;5;12mlike[39m[38;5;12m [39m[38;5;12mnative[39m[38;5;12m [39m[38;5;12mScala[39m[38;5;12m [39m[38;5;12mor[39m[38;5;12m [39m[38;5;12mJava[39m[38;5;12m [39m[38;5;12mcollection[39m[38;5;12m [39m[38;5;12mtransformations[39m[38;5;12m [39m[38;5;12mand[39m[38;5;12m [39m[38;5;12mexecute[39m[38;5;12m [39m[38;5;12mthem[39m[38;5;12m [39m[38;5;12mon[39m[38;5;12m [39m[38;5;12ma[39m[38;5;12m [39m[38;5;12mnumber[39m[38;5;12m [39m[38;5;12mof[39m[38;5;12m [39m[38;5;12mwell-known[39m[38;5;12m [39m[38;5;12mdistributed[39m[38;5;12m [39m
[38;5;12mMapReduce[39m[38;5;12m [39m[38;5;12mplatforms,[39m[38;5;12m [39m[38;5;12mincluding[39m[38;5;12m [39m[38;5;12mStorm[39m[38;5;12m [39m[38;5;12mand[39m[38;5;12m [39m[38;5;12mScalding.[39m

[38;2;255;187;0m[4mData Pipeline[0m

[38;5;12m- [39m[38;5;14m[1mApache Kafka[0m[38;5;12m (https://github.com/apache/kafka) [39m[38;5;14m[1mScala/Java[0m[38;5;12m  - distributed, partitioned, replicated commit log service, which provides the functionality of a messaging system, but with a unique design.[39m
[38;5;12m- [39m[38;5;14m[1mApache Pulsar[0m[38;5;12m (https://github.com/apache/incubator-pulsar) [39m[38;5;14m[1mJava[0m[38;5;12m  - distributed pub-sub messaging platform with a very flexible messaging model and an intuitive client API.[39m
[38;5;12m- [39m[38;5;14m[1mApache RocketMQ[0m[38;5;12m (https://github.com/apache/rocketmq) [39m[38;5;14m[1mJava[0m[38;5;12m  - distributed messaging and streaming platform with low latency, high performance and reliability, trillion-level capacity and flexible scalability.[39m
[38;5;12m-[39m[38;5;12m [39m[38;5;14m[1mbrooklin[0m[38;5;12m [39m[38;5;12m(https://github.com/linkedin/Brooklin/)[39m[38;5;12m [39m[38;5;14m[1mJava[0m[38;5;12m  [39m[38;5;12m-[39m[38;5;12m [39m[38;5;12ma[39m[38;5;12m [39m[38;5;12mdistributed[39m[38;5;12m [39m[38;5;12msystem[39m[38;5;12m [39m[38;5;12mintended[39m[38;5;12m [39m[38;5;12mfor[39m[38;5;12m [39m[38;5;12mstreaming[39m[38;5;12m [39m[38;5;12mdata[39m[38;5;12m [39m[38;5;12mbetween[39m[38;5;12m [39m[38;5;12mvarious[39m[38;5;12m [39m[38;5;12mheterogeneous[39m[38;5;12m [39m[38;5;12msource[39m[38;5;12m [39m[38;5;12mand[39m[38;5;12m [39m[38;5;12mdestination[39m[38;5;12m [39m[38;5;12msystems[39m[38;5;12m [39m[38;5;12mwith[39m[38;5;12m [39m[38;5;12mhigh[39m[38;5;12m [39m[38;5;12mreliability[39m[38;5;12m [39m[38;5;12mand[39m[38;5;12m [39m[38;5;12mthroughput[39m[38;5;12m [39m[38;5;12mat[39m[38;5;12m [39m[38;5;12mscale[39m[38;5;12m [39m[38;5;12mfrom[39m[38;5;12m [39m[38;5;12mLinkedin[39m[38;5;12m [39m
[38;5;12m(replaced[39m[38;5;12m [39m[38;5;12mdatabus).[39m
[38;5;12m- [39m[38;5;14m[1mcamus[0m[38;5;12m (https://github.com/linkedin/camus) [39m[38;5;14m[1mJava[0m[38;5;12m  - Linkedin's Kafka -> HDFS pipeline.[39m
[38;5;12m- [39m[38;5;14m[1mdatabus[0m[38;5;12m (https://github.com/linkedin/databus) [39m[38;5;14m[1mJava[0m[38;5;12m  - Linkedin's source-agnostic distributed change data capture system.[39m
[38;5;12m- [39m[38;5;14m[1mflume[0m[38;5;12m (https://github.com/apache/flume) [39m[38;5;14m[1mJava[0m[38;5;12m  - distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data.[39m
[38;5;12m- [39m[38;5;14m[1mfluvio[0m[38;5;12m (https://github.com/infinyon/fluvio) [39m[38;5;14m[1mRust/WASM[0m[38;5;12m  - Real-time programmable data streaming platform with in-line computation capabilities.[39m
[38;5;12m- [39m[38;5;14m[1mGazette[0m[38;5;12m (https://github.com/gazette/core) [39m[38;5;14m[1mgolang[0m[38;5;12m  - Distributed streaming infrastructure built on cloud storage which makes it easy to mix and match batch and streaming paradigms.[39m
[38;5;12m- [39m[38;5;14m[1mLogDevice[0m[38;5;12m (https://logdevice.io/) [39m[38;5;14m[1mC++[0m[38;5;12m  - a high-performant distributed system by Facebook for streaming and storing sequential data, using a log structure.[39m
[38;5;12m- [39m[38;5;14m[1mmetaq[0m[38;5;12m (https://github.com/killme2008/Metamorphosis) [39m[38;5;14m[1mJava[0m[38;5;12m  - Taobao's high available, high performance distributed messaging system[39m
[38;5;12m- [39m[38;5;14m[1mNATS streaming[0m[38;5;12m (https://github.com/nats-io/nats-streaming-server) [39m[38;5;14m[1mGo[0m[38;5;12m  - fast disk-backed messaging solution[39m
[38;5;12m- [39m[38;5;14m[1mnsq[0m[38;5;12m (https://github.com/nsqio/nsq) [39m[38;5;14m[1mGo[0m[38;5;12m  - realtime distributed messaging platform designed to operate at scale, handling billions of messages per day.[39m
[38;5;12m- [39m[38;5;14m[1mRedpanda[0m[38;5;12m (https://github.com/redpanda-data/redpanda) [39m[38;5;14m[1mC++[0m[38;5;12m  - Redpanda is Kafka compatible, ZooKeeper-free, JVM-free and source available.[39m
[38;5;12m- [39m[38;5;14m[1mRudderStack[0m[38;5;12m (https://github.com/rudderlabs/rudder-server) [39m[38;5;14m[1mGo[0m[38;5;12m  - an open source customer data infrastructure (segment, mparticle alternative).[39m
[38;5;12m- [39m[38;5;14m[1msuro[0m[38;5;12m (https://github.com/Netflix/suro) [39m[38;5;14m[1mJava[0m[38;5;12m  - data pipeline service for collecting, aggregating, and dispatching large volume of application events including log data.[39m
[38;5;12m-[39m[38;5;12m [39m[38;5;14m[1mStreamSets[0m[38;5;14m[1m [0m[38;5;14m[1mData[0m[38;5;14m[1m [0m[38;5;14m[1mCollector[0m[38;5;12m [39m[38;5;12m(https://github.com/streamsets/datacollector-oss)[39m[38;5;12m [39m[38;5;14m[1mJava[0m[38;5;12m  [39m[38;5;12m-[39m[38;5;12m [39m[38;5;12mcontinuous[39m[38;5;12m [39m[38;5;12mbig[39m[38;5;12m [39m[38;5;12mdata[39m[38;5;12m [39m[38;5;12mingestion[39m[38;5;12m [39m[38;5;12minfrastructure[39m[38;5;12m [39m[38;5;12mthat[39m[38;5;12m [39m[38;5;12mreads[39m[38;5;12m [39m[38;5;12mfrom[39m[38;5;12m [39m[38;5;12mand[39m[38;5;12m [39m[38;5;12mwrites[39m[38;5;12m [39m[38;5;12mto[39m[38;5;12m [39m[38;5;12ma[39m[38;5;12m [39m[38;5;12mlarge[39m[38;5;12m [39m[38;5;12mnumber[39m[38;5;12m [39m[38;5;12mof[39m[38;5;12m [39m[38;5;12mend-points,[39m[38;5;12m [39m[38;5;12mincluding[39m[38;5;12m [39m[38;5;12mS3,[39m[38;5;12m [39m[38;5;12mJDBC,[39m[38;5;12m [39m[38;5;12mHadoop,[39m[38;5;12m [39m[38;5;12mKafka,[39m[38;5;12m [39m[38;5;12mCassandra[39m[38;5;12m [39m
[38;5;12mand[39m[38;5;12m [39m[38;5;12mmany[39m[38;5;12m [39m[38;5;12mothers.[39m

[38;2;255;187;0m[4mOnline Machine Learning [0m

[38;5;12m- [39m[38;5;14m[1mApache Samoa[0m[38;5;12m (https://github.com/apache/incubator-samoa) [39m[38;5;14m[1mJava[0m[38;5;12m  - distributed streaming machine learning (ML) framework that contains a programing abstraction for distributed streaming ML algorithms.[39m
[38;5;12m- [39m[38;5;14m[1mDataSketches[0m[38;5;12m (https://github.com/DataSketches/sketches-core) [39m[38;5;14m[1mJava[0m[38;5;12m  - sketches library from Yahoo!.[39m
[38;5;12m- [39m[38;5;14m[1mNumalogic[0m[38;5;12m  (https://github.com/numaproj/numalogic) [39m[38;5;14m[1mPython[0m[38;5;12m  - Collection of ML models and libraries for real-time anomaly detection and forecasting on time series data. Built on Numaflow, a K8s native stream processing platform[39m
[38;5;12m- [39m[38;5;14m[1mRiver[0m[38;5;12m (https://github.com/online-ml/river) [39m[38;5;14m[1mPython[0m[38;5;12m  - online machine learning library.[39m
[38;5;12m- [39m[38;5;14m[1mstreamDM[0m[38;5;12m (https://github.com/huawei-noah/streamDM) [39m[38;5;14m[1mScala[0m[38;5;12m  - mining Big Data streams using Spark Streaming from Huawei.[39m
[38;5;12m- [39m[38;5;14m[1mStreamingBandit[0m[38;5;12m (https://github.com/Nth-iteration-labs/streamingbandit) [39m[38;5;14m[1mPython[0m[38;5;12m  - Provides a webserver to quickly setup and evaluate possible solutions to contextual multi-armed bandit (cMAB) problems.[39m
[38;5;12m- [39m[38;5;14m[1mStormCV[0m[38;5;12m (https://github.com/sensorstorm/StormCV) [39m[38;5;14m[1mJava[0m[38;5;12m  - enables the use of Apache Storm for video processing by adding computer vision (CV) specific operations and data model.[39m
[38;5;12m- [39m[38;5;14m[1mtrident-ml[0m[38;5;12m (https://github.com/pmerienne/trident-ml) [39m[38;5;14m[1mJava[0m[38;5;12m  - realtime online machine learning library based on Trident.[39m
[38;5;12m- [39m[38;5;14m[1myurita[0m[38;5;12m (https://github.com/paypal/yurita) [39m[38;5;14m[1mScala[0m[38;5;12m  - Anomaly detection framework built on Spark Structured Streaming from Paypal.[39m

[38;2;255;187;0m[4mStreaming SQL[0m

[38;5;12m- [39m[38;5;14m[1mpipelinedb[0m[38;5;12m (https://github.com/pipelinedb/pipelinedb) [39m[38;5;14m[1mC[0m[38;5;12m  - An open-source relational database that runs SQL queries continuously on streams, incrementally storing results in tables.[39m
[38;5;12m- [39m[38;5;14m[1msquall[0m[38;5;12m (https://github.com/epfldata/squall) [39m[38;5;14m[1mJava[0m[38;5;12m  - Squall executes SQL queries on top of Storm for doing online processing.[39m
[38;5;12m- [39m[38;5;14m[1mStreamCQL[0m[38;5;12m (https://github.com/Zhiqiang-He/StreamCQL) [39m[38;5;14m[1mJava[0m[38;5;12m  - Continuous Query Language on RealTime Computation System.[39m
[38;5;12m- [39m[38;5;14m[1mksqlDB[0m[38;5;12m (https://github.com/confluentinc/ksql) [39m[38;5;14m[1mJava[0m[38;5;12m  - A cloud-native, source-available [39m[38;5;14m[1mdatabase[0m[38;5;12m (https://ksqldb.io/) purpose-built for stream processing applications[39m
[38;5;12m- [39m[38;5;14m[1mMaterialize[0m[38;5;12m (https://materialize.com) [39m[38;5;14m[1mRust[0m[38;5;12m  - A source-available streaming SQL engine for maintaining materialized views on data from message brokers and databases.[39m
[38;5;12m-[39m[38;5;12m [39m[38;5;14m[1mSiddhi[0m[38;5;12m [39m[38;5;12m(https://github.com/siddhi-io/siddhi)[39m[38;5;12m [39m[38;5;14m[1mJava[0m[38;5;12m  [39m[38;5;12m-[39m[38;5;12m [39m[38;5;12mA[39m[38;5;12m [39m[38;5;12mcloud[39m[38;5;12m [39m[38;5;12mnative[39m[38;5;12m [39m[38;5;12mStreaming[39m[38;5;12m [39m[38;5;12mand[39m[38;5;12m [39m[38;5;12mComplex[39m[38;5;12m [39m[38;5;12mEvent[39m[38;5;12m [39m[38;5;12mProcessing[39m[38;5;12m [39m[38;5;12mengine[39m[38;5;12m [39m[38;5;12mthat[39m[38;5;12m [39m[38;5;12munderstands[39m[38;5;12m [39m[38;5;12mStreaming[39m[38;5;12m [39m[38;5;12mSQL[39m[38;5;12m [39m[38;5;12mqueries[39m[38;5;12m [39m[38;5;12min[39m[38;5;12m [39m[38;5;12morder[39m[38;5;12m [39m[38;5;12mto[39m[38;5;12m [39m[38;5;12mcapture[39m[38;5;12m [39m[38;5;12mevents[39m[38;5;12m [39m[38;5;12mfrom[39m[38;5;12m [39m[38;5;12mdiverse[39m[38;5;12m [39m[38;5;12mdata[39m[38;5;12m [39m[38;5;12msources,[39m[38;5;12m [39m[38;5;12mprocess[39m[38;5;12m [39m[38;5;12mthem,[39m[38;5;12m [39m[38;5;12mdetect[39m[38;5;12m [39m
[38;5;12mcomplex[39m[38;5;12m [39m[38;5;12mconditions,[39m[38;5;12m [39m[38;5;12mand[39m[38;5;12m [39m[38;5;12mpublish[39m[38;5;12m [39m[38;5;12moutput[39m[38;5;12m [39m[38;5;12mto[39m[38;5;12m [39m[38;5;12mvarious[39m[38;5;12m [39m[38;5;12mendpoints[39m[38;5;12m [39m[38;5;12min[39m[38;5;12m [39m[38;5;12mreal[39m[38;5;12m [39m[38;5;12mtime.[39m
[38;5;12m- [39m[38;5;14m[1mProton[0m[38;5;12m (https://github.com/timeplus-io/proton) [39m[38;5;14m[1mC++[0m[38;5;12m  - A unified streaming and historical data analytics database in a single binary, powered by ClickHouse.[39m

[38;2;255;187;0m[4mBenchmark[0m

[38;5;12m- [39m[38;5;14m[1mstorm-perf-test[0m[38;5;12m (https://github.com/yahoo/storm-perf-test) [39m[38;5;14m[1mJava[0m[38;5;12m  - a simple storm performance/stress test.[39m
[38;5;12m- [39m[38;5;14m[1mstreaming-benchmarks[0m[38;5;12m (https://github.com/yahoo/streaming-benchmarks) [39m[38;5;14m[1mJava[0m[38;5;12m  - Benchmarks for Low Latency (Streaming) solutions including Apache Storm, Apache Spark, Apache Flink, etc.[39m
[38;5;12m- [39m[38;5;14m[1mflotilla[0m[38;5;12m (https://github.com/tylertreat/Flotilla) [39m[38;5;14m[1mGo[0m[38;5;12m  - Automated message queue orchestration for scaled-up benchmarking.[39m

[38;2;255;187;0m[4mToolkit[0m

[38;5;12m- [39m[38;5;14m[1makka[0m[38;5;12m (https://github.com/akka/akka) [39m[38;5;14m[1mScala[0m[38;5;12m  - toolkit and runtime for building highly concurrent, distributed, and resilient message-driven application on the JVM.[39m
[38;5;12m- [39m[38;5;14m[1mApache Pekko[0m[38;5;12m (https://github.com/apache/incubator-pekko) [39m[38;5;14m[1mScala, Java[0m[38;5;12m  - Fork of Akka 2.6.x, prior to the Akka project's adoption of the Business Source License.[39m
[38;5;12m- [39m[38;5;14m[1mpulsar[0m[38;5;12m (https://github.com/quantmind/pulsar/) [39m[38;5;14m[1mPython[0m[38;5;12m  - Actor based event driven concurrent framework for Python.[39m
[38;5;12m- [39m[38;5;14m[1maeron[0m[38;5;12m (https://github.com/real-logic/Aeron) [39m[38;5;14m[1mJava/C++[0m[38;5;12m  - efficient reliable unicast and multicast message transport.[39m
[38;5;12m- [39m[38;5;14m[1mStreamFlow[0m[38;5;12m (https://github.com/lmco/streamflow) [39m[38;5;14m[1mJava[0m[38;5;12m  - stream processing tool designed to help build and monitor processing workflows.[39m
[38;5;12m- [39m[38;5;14m[1msamza-luwak[0m[38;5;12m (https://github.com/romseygeek/samza-luwak) [39m[38;5;14m[1mJava[0m[38;5;12m  - uses Luwak, a stored-query engine built on Lucene, to implement full-text search on streams.[39m
[38;5;12m- [39m[38;5;14m[1mStreamdal[0m[38;5;12m (https://streamdal.com) [39m[38;5;14m[1mGo/Node.js/Python[0m[38;5;12m  - A tool to embed privacy controls in your application code to detect PII as it enters and leaves your systems, preventing it from reaching unintended data streams or pipelines.[39m
[38;5;12m- [39m[38;5;14m[1mTurbine[0m[38;5;12m (https://github.com/Netflix/Turbine) [39m[38;5;14m[1mJava[0m[38;5;12m  - tool for aggregating streams of Server-Sent Event (SSE) JSON data into a single stream.[39m
[38;5;12m- [39m[38;5;14m[1mNussknacker[0m[38;5;12m (https://github.com/TouK/nussknacker) [39m[38;5;14m[1mScala[0m[38;5;12m  - A visual tool to define and run real-time decision algorithms.[39m

[38;2;255;187;0m[4mClosed Source[0m

[38;5;12m- [39m[38;5;14m[1mAmazon Kinesis Streams[0m[38;5;12m (https://aws.amazon.com/kinesis/) [39m[38;5;14m[1mJava[0m[38;5;12m  - real-time, fully managed and scalable data stream engine provided by AWS. [39m
[38;5;12m- [39m[38;5;14m[1mAzure Stream Analytics[0m[38;5;12m (https://azure.microsoft.com/en-us/services/stream-analytics/) [39m[38;5;14m[1m.NET[0m[38;5;12m  a massively scalable, fully managed, real-time, data stream engine provided by Microsoft Azure.[39m
[38;5;12m- [39m[38;5;14m[1mCloud Dataflow[0m[38;5;12m (https://cloud.google.com/dataflow/)[39m[38;5;14m[1mJava, Python, SQL, Scala[0m[38;5;12m  - Google's managed stream and batch data processing engine. Supports running Beam pipelines.[39m
[38;5;12m- [39m[38;5;14m[1mconcord[0m[38;5;12m (https://www.slideshare.net/concord-io/may-2016-data-by-the-bay-concord-simple-flexible-stream-processing-on-apache-mesos) [39m[38;5;14m[1mC++[0m[38;5;12m  - a distributed stream processing framework built in C++ on top of Apache.[39m
[38;5;12m-[39m[38;5;12m [39m[38;5;14m[1mIBM[0m[38;5;14m[1m [0m[38;5;14m[1mStreams[0m[38;5;12m [39m[38;5;12m(https://www.ibm.com/analytics/us/en/technology/stream-computing/)[39m[38;5;12m [39m[38;5;14m[1mPython/Java/Scala[0m[38;5;12m  [39m[38;5;12m-[39m[38;5;12m [39m[38;5;12mplatform[39m[38;5;12m [39m[38;5;12mfor[39m[38;5;12m [39m[38;5;12mdistributed[39m[38;5;12m [39m[38;5;12mprocessing[39m[38;5;12m [39m[38;5;12mand[39m[38;5;12m [39m[38;5;12mreal-time[39m[38;5;12m [39m[38;5;12manalytics.[39m[38;5;12m [39m[38;5;12mProvides[39m[38;5;12m [39m[38;5;12mtoolkits[39m[38;5;12m [39m[38;5;12mfor[39m[38;5;12m [39m[38;5;12madvanced[39m[38;5;12m [39m[38;5;12manalytics[39m[38;5;12m [39m[38;5;12mlike[39m[38;5;12m [39m[38;5;12mgeospatial,[39m[38;5;12m [39m[38;5;12mtime[39m[38;5;12m [39m[38;5;12mseries,[39m[38;5;12m [39m
[38;5;12metc.[39m[38;5;12m [39m[38;5;12mout[39m[38;5;12m [39m[38;5;12mof[39m[38;5;12m [39m[38;5;12mthe[39m[38;5;12m [39m[38;5;12mbox.[39m
[38;5;12m- [39m[38;5;14m[1mjubatus[0m[38;5;12m (http://jubat.us/en/) [39m[38;5;14m[1mC++[0m[38;5;12m  - distributed processing framework and streaming machine learning library.[39m
[38;5;12m- [39m[38;5;14m[1mmillwheel[0m[38;5;12m (http://research.google.com/pubs/pub41378.html) - framework for building low-latency data-processing applications that is widely used at Google.[39m
[38;5;12m- [39m[38;5;14m[1mNVIDIA Deep Stream[0m[38;5;12m (https://developer.nvidia.com/deepstream-sdk) [39m[38;5;14m[1mPython/C/C++[0m[38;5;12m  - a platform for real-time image, video and audio processing, preferably using on edge devices or cloud.[39m

[38;2;255;187;0m[4mReadings[0m

[38;5;12m1. [39m[38;5;14m[1mIn-Stream Big Data Processing[0m[38;5;12m (https://highlyscalable.wordpress.com/2013/08/20/in-stream-big-data-processing/)[39m
[38;5;12m2. [39m[38;5;14m[1mThe world beyond batch: Streaming 101[0m[38;5;12m (http://radar.oreilly.com/2015/08/the-world-beyond-batch-streaming-101.html) by Tyler Akidau. [39m
[38;5;12m3. [39m[38;5;14m[1mReal Time Analytics: Algorithms and Systems (VLDB 2015)[0m[38;5;12m (http://www.vldb.org/pvldb/vol8/p2040-Kejariwal.pdf)[39m
[38;5;12m4. [39m[38;5;14m[1mGrokking Streaming Systems[0m[38;5;12m (https://www.manning.com/books/grokking-streaming-systems) by Josh Fischer & Ning Wang[39m
[38;5;12m5. [39m[38;5;14m[1mStreaming Systems: The What, Where, When, and How of Large-Scale Data Processing[0m[38;5;12m (https://www.oreilly.com/library/view/streaming-systems/9781491983867/) by Reuven Lax, Slava Chernyak, and Tyler Akidau[39m
[38;5;12m6. [39m[38;5;14m[1mData Pipelines with Apache Airflow[0m[38;5;12m (https://www.manning.com/books/data-pipelines-with-apache-airflow) by Bas P. Harenslak and Julian Rutger de Ruiter[39m

[38;2;255;187;0m[4mLicense[0m

[38;5;12m![39m[38;5;14m[1mCreative Commons License[0m[38;5;12m (https://i.creativecommons.org/l/by-sa/4.0/80x15.png)[39m

[38;5;12mLicensed under a [39m[38;5;14m[1mCreative Commons Attribution-ShareAlike 4.0 International License[0m[38;5;12m (http://creativecommons.org/licenses/by-sa/4.0/)[39m
