A curated list of awesome system integration software, patterns and resources.
System integration is the process of linking together different IT systems (components) to functionally cooperate as a whole.
API Management solutions offer a comprehensive approach to handling the entire API lifecycle. They enable developers to create, publish, secure, and monitor APIs, ensuring efficient and reliable communication between systems. With features such as authentication, rate limiting, and analytics, these tools provide centralized control and enhanced security, making it easier to manage and scale APIs across different platforms and environments. - Akana API Management Platform - A comprehensive platform that accelerates digital transformation, manages the full API lifecycle, and ensures compliance while deploying across multiple clouds. - Anypoint Platform - A single, unified platform from Mulesoft that combines API management and integration capabilities, allowing you to design, build, and manage APIs with ease. - Amazon API Management - A secure and scalable API management solution that enables you to monitor and manage APIs in a flexible and efficient way. - Amplify API Management Platform - An open platform for API management from Axway, which provides comprehensive API management capabilities and allows you to build, run, and manage APIs with ease. - Apigee - A leading API management solution from Google that provides comprehensive API management capabilities and helps you create, secure, and scale APIs with ease. - Azure API Management - A hybrid, multicloud management platform for APIs across all environments, which enables you to manage APIs with ease, secure them, and gain insights into their performance. - DigitMarket API Manager - A complete package from Torry Harris that helps you manage your APIs and turn them into tools that can drive your business forward. - Gravitee.io API Management (⭐133) - A flexible, lightweight, and open-source API management solution that provides comprehensive API management capabilities and helps you manage your APIs with ease. - IBM API Connect - A complete, intuitive, and scalable API platform that allows you to create, expose, manage, and monetize APIs across clouds. - Kong Enterprise - A comprehensive service connectivity platform that provides a range of API management capabilities and allows you to manage APIs, microservices, and service meshes with ease. - Layer7 API Management - A continuous API management solution from Broadcom that provides comprehensive API management capabilities and helps you manage APIs throughout their lifecycle. - Red Hat 3scale API Management - A cloud-native API management solution that simplifies the development of APIs and makes them more flexible. - Sensedia API Management - A full lifecycle API management platform that provides comprehensive API management capabilities and helps you manage APIs throughout their lifecycle. - TIBCO Cloud Mashery - A cloud-native API platform from TIBCO that enables you to deploy APIs anywhere and manage them from everywhere. - Tyk API Management - A powerful API management solution that allows you to connect, build, and control APIs with ease. - webMethods API - A comprehensive API management solution from Software AG that provides full lifecycle API management capabilities and enables you to manage APIs in both on-premises and cloud environments. - WSO2 API Manager (⭐801) - A fully open-source API management platform that provides comprehensive API management capabilities and allows you to manage APIs with ease.
⬆ back to top ### API Design API design, documentation, and life-cycle automation tools streamline the process of creating, maintaining, and evolving APIs. These tools offer efficient means to design consistent and scalable APIs, generate comprehensive documentation, and automate various stages of the API life-cycle, enhancing developer experience and promoting API adoption. - Dredd (⭐4.1k) - Validate API description docs against backend implementation using this language-agnostic CLI tool. - OpenAPI-GUI (⭐1.3k) - Create and validate OpenAPI specs with ease using this intuitive graphical user interface. - OpenAPI Diff (⭐743) - Compare OpenAPI specs with version control and visualize the differences in HTML or Markdown format. - OpenAPI Generator (⭐19k) - Automate the creation of API client libraries, server stubs, documentation and config files with this powerful OpenAPI Spec tool. - OpenAPI Style Validator (⭐194) - Ensure that your OpenAPI specs meet your organization’s standards with this flexible and customizable style validator. - Spectral (⭐2.2k) - Detect and fix errors in your JSON/YAML files using this linter tool that supports OpenAPI 3.0 & 2.0 and AsyncAPI. - Swagger Editor (⭐8.6k) - Create, describe, and document your API with ease using this open source editor built specifically for OpenAPI-based APIs. - Zally (⭐874) - Ensure the quality of your OpenAPI specs with this linter tool that provides extensive analysis and feedback.
⬆ back to top ### API Documentation Explore a collection of tools and platforms designed to create, maintain, and present clear, concise, and user-friendly documentation for APIs. These resources facilitate seamless collaboration between developers and enable effective communication of API functionalities and specifications. - DapperDox (⭐392) - An open-source API documentation generator and server for OpenAPI/Swagger specifications, with customizable documentation, automated updates, and easy sharing. - OpenAPI Explorer (⭐281) - A tool for generating user interfaces from OpenAPI specifications, making it easier for software engineers to visualize and interact with APIs. - RapiDoc (⭐1.5k) - A tool that generates customizable, interactive API documentation from OpenAPI Specification, with a range of design options. - Redoc (⭐22k) - An open-source tool for generating documentation from OpenAPI (fka Swagger) definitions, with customizable themes, language support, and branding. - Slate (⭐35k) - A powerful tool that creates static API documentation from markdown, with API reference docs and code samples. - Stoplight Elements (⭐1.4k) - Beautiful API documentation powered by OpenAPI and Markdown. Use these UI components to create API reference documentation, or more complete documentation with Markdown articles covering tutorials, how-to guides, etc. - SwaggerHub - An integrated API design and documentation platform built for teams, with versioning, testing automation, and integration with popular API tools. - Swagger UI (⭐25k) - An intuitive tool for exploring and interacting with APIs, with API specification visualization, endpoint exploration, and built-in interactive console. - SpectaQL (⭐1k) - A Node.js library that generates static documentation for a GraphQL schema from various sources, including live endpoint, query result, and SDL files.
⬆ back to top ### API Gateway API Gateways act as intermediaries between client applications and backend services, enabling key features such as request routing, authentication, rate limiting, and caching. They simplify the process of managing, securing, and monitoring APIs by providing a unified entry point for various services. - Apinto (⭐1.2k) - A Golang-based microservice gateway that provides dynamic routing, service orchestration, multi-tenancy management, and API access control. - Ambassador Edge Stack - A Kubernetes-native API Gateway that helps manage APIs at scale with its customizable routing, authentication, and rate-limiting features. - Apache APISIX (⭐13k) - A dynamic, real-time, and high-performance API Gateway that leverages Apache plugins for enhanced functionality. - Gloo Edge (⭐3.9k) - An Envoy Proxy-based API Gateway that provides advanced traffic management, security, and observability features for modern microservices architectures. - Kong API Gateway (⭐37k) - A cloud-native, platform-agnostic, and scalable API Gateway that supports plugins for easy integration with third-party services. - KrakenD API Gateway (⭐1.7k) - An ultra-high performance API Gateway that leverages middlewares for fast and efficient API management. - Ocelot (⭐8.1k) - A .NET API Gateway that provides a simple and elegant way to manage APIs with its intuitive routing and middleware configuration. - Spring Cloud Gateway (⭐4.3k) - An API Gateway built on top of the Spring ecosystem, providing advanced routing and filtering features for Spring applications. - Traefik (⭐47k) - A modern, open-source API Gateway that is designed to handle dynamic container environments such as Kubernetes, Docker Swarm, and Mesos. It provides load balancing, SSL/TLS termination, rate limiting, circuit breaking, and more. - Tyk API Gateway (⭐9.2k) - An open source Enterprise API Gateway that supports REST, GraphQL, TCP, and gRPC protocols. It provides advanced API management features such as rate limiting, access control, and analytics.
⬆ back to top ### API Testing The API testing tools section provides a list of software tools and frameworks that developers can use to test REST APIs and message brokers. This section includes GUI clients for testing REST APIs and message brokers, mocking tools for simulating API responses, as well as various testing tools and frameworks to automate testing processes. - API clients - Advanced REST Client (⭐1.3k) - An open-source API tool designed for testing and debugging RESTful web services. - curl (⭐34k) - Command-line tool for transferring data with URLs over various protocols. - curlie (⭐2.6k) - Frontend to curl with the ease of use of httpie without compromising on performance and features. - HTTPie (⭐31k) - User-friendly command-line HTTP client that simplifies HTTP request/response inspection and manipulation. - Insomnia (⭐33k) - Cross-platform HTTP and GraphQL client designed to make API testing and debugging easier. - Postman - Comprehensive tool for simplifying API development and testing processes with features such as making HTTP requests, generating mock servers, and creating API documentation. - resty (⭐2.6k) - A Lua framework for building HTTP clients that provides a shell interface for interacting with REST services. - SoapUI (⭐1.4k) - Open-source web service testing application designed for testing SOAP and REST web services. - MQ clients - JMSToolBox (⭐190) - Universal JMS client designed to interact with a wide range of JMS-based message brokers. - kcat (⭐5.2k) - Generic non-JVM producer and consumer for Apache Kafka, functioning as a netcat for Kafka. - MQTT Explorer (⭐2.7k) - All-round MQTT client with a structured topic overview, suitable for debugging and testing MQTT-based systems. - Offset Explorer - GUI application for managing and using Apache Kafka clusters, formerly known as Kafka Tool. - Service Bus Explorer (⭐1.9k) - GUI app for Azure Service Bus, offering advanced features like import/export functionality and the ability to test topic, queues, subscriptions, relay services, notification hubs, and events hubs. - Mocking tools - Imposter (⭐341) - Mock server for REST APIs, OpenAPI (and Swagger) specifications, SOAP web services (and WSDL files), Salesforce and HBase APIs. - Microcks (⭐1.1k) - Open-source Kubernetes-native tool for API mocking and testing, supporting AsyncAPI, OpenAPI, Postman Collections, etc. - Mockable - Simple configurable service to mock out RESTful API or SOAP web-services. - Mockoon (⭐5.8k) - Easy-to-use tool for designing and running mock REST APIs. - MockServer (⭐4.4k) - Easy mocking of any system you integrate with via HTTP or HTTPS. - Mocky (⭐2k) - Free and unlimited online service for generating custom HTTP responses. - Prism (⭐3.9k) - Open-source HTTP mock server that can mimic your API’s behavior as if you already built it. - WireMock (⭐6k) - Mock your APIs for fast, robust, and comprehensive testing. - Testing tools and frameworks - Apache JMeter (⭐7.9k) - Open-source Java-based load testing tool that can be used to test the performance and scalability of web applications, databases, and other network services. - Gatling (⭐6.2k) - An open-source load testing tool that allows users to write and run load tests as code, providing detailed performance reports to optimize system performance. - Karate (⭐7.8k) - Open-source tool that combines API test-automation, mocks, performance-testing, and even UI automation into a single, unified framework. - Pyresttest (⭐1.1k) - Java DSL for easy testing of REST services. - REST Assured (⭐6.7k) - Java DSL for easy testing of REST services. - Schemathesis (⭐2k) - Python library for property-based testing of API schemas. - Taurus (⭐1.9k) - An open-source framework for continuous testing that simplifies test execution and management through a user-friendly interface and integrates with various testing tools and frameworks.
⬆ back to top ### BRE Business Rules Engines and Business Rules Management Systems (BRMS) are software systems that allow organizations to define, manage, and execute business rules. These rules are used to automate decision-making processes, enforce business policies, and ensure regulatory compliance. A BRMS provides a way for organizations to centrally manage their business rules and make changes to them quickly and easily, without requiring changes to the underlying code. It is a valuable tool for businesses that need to be agile and respond quickly to changing market conditions. - Drools (⭐5.6k) - Open-source tool that supports Decision Model and Notation (DMN) models at Conformance level 3 with an Eclipse IDE plugin for core development. - Easy Rules (⭐4.7k) - Simple Java rules engine for straightforward rule processing. - NxBRE (⭐129) - Open-source rule engine specifically designed for the .NET platform. - OpenL Tablets (⭐143) - Open-source business rules engine (BRE, BRMS) and decision management system that allows users to define, manage, and execute business rules and decision tables.
⬆ back to top ### BPM BPM solutions are software tools that help businesses streamline and automate their operational processes to improve efficiency and productivity. These solutions typically provide features such as process modeling, workflow management, task automation, and reporting. They can be used to automate a wide range of processes, from simple tasks like data entry to complex workflows involving multiple departments and stakeholders. - Appian BPM Suite - A platform that combines low-code development with process management, allowing both IT and citizen developers to construct process-centric and case-centric applications. - Genpack Cora SeQuence - A tool for orchestrating business processes, driving business agility and improving cost efficiency. - IBM Business Automation Workflow - A comprehensive business process management platform for developing and administering business applications. - Oracle BPM Suite - An integrated environment for developing, administering, and using business applications that are centered around business processes. - Pega Platform - A platform for business process management and robotic process automation (RPA) with advanced workforce analytics. - Red Hat Process Automation Manager - Application development platform that enables developers and business experts to create cloud-native applications that automate business operations. - SAP Process Orchestration - A business process management offering and application infrastructure. - TIBCO BPM Enterprise - Business process management platform that offers capabilities for process automation, documentation, human capital management, process patterns, and predictive operations analytics.
⬆ back to top ### Data Mapping Solution Data Mapping tools are used to define and transform data between different systems, applications, and formats. These tools allow for the mapping of data from source to target, allowing for data transformation and integration. This section covers different data mapping solutions that can be used to facilitate data mapping and transformation processes for different use cases. - AltasMap (⭐179) - A web-based data mapping solution that simplifies integration between Java, XML, CSV, and JSON data sources. Its interactive user interface makes configuring integrations easy. - Altova MapForce - A graphical data mapping tool that can handle any-to-any conversion and integration tasks. It simplifies complex mapping tasks, enabling software engineers to complete projects faster. - JOLT (⭐1.4k) - A Java-based JSON-to-JSON transformation library that uses a JSON document as the “specification” for the transform. JOLT provides a simple way for software engineers to transform JSON data. - JSLT (⭐604) - A complete query and transformation language for JSON that is inspired by jq, XPath, and XQuery. With JSLT, software engineers can quickly and easily transform JSON data to meet their needs.
⬆ back to top ### ESB Enterprise Service Bus (ESB) solutions facilitate the integration of disparate systems and applications by providing a communication layer that allows them to exchange information. ESBs offer a range of capabilities such as message routing, transformation, protocol conversion, and data mediation. They typically support various messaging patterns and communication protocols and provide a centralized platform for managing and monitoring message traffic. - Anypoint Platform - A comprehensive API management and integration platform that simplifies connecting applications from Mulesoft. - Apache ServiceMix - An open-source integration container that combines the functionality of Apache ActiveMQ, Camel, CXF, and Karaf, providing a flexible solution. - ArcESB - A versatile integration platform that seamlessly synchronizes data across applications, integrates with partners, and provides data accessibility. - IBM App Connect - An integration platform that can connect applications, irrespective of the message formats or protocols they use, formerly known as IBM Integration Bus. - NServiceBus (⭐2k) - A .NET-based service bus that offers an intuitive developer-friendly environment. - Oracle Service Bus - An integration platform that connects, virtualizes, and manages interactions between services and applications. - Oracle SOA Suite - A platform that enables system developers to set up and manage services and to orchestrate them into composite applications and business processes. - Red Hat Fuse - A cloud-native integration platform that supports distributed integration capabilities. - Software AG webMethods Integration Server - An integration platform that enables faster integration of any application. - TIBCO BusinessWorks - A platform that implements enterprise patterns for hybrid integrations. - UltraESB - An ESB that supports zero-copy proxying for extreme performance utilizing Direct Memory Access and Non-Blocking IO. - WSO2 Enterprise Integrator (⭐362) - An API-centric, cloud-native, and distributed integration platform designed to provide a robust solution for software engineers.
⬆ back to top ### ETL ETL (Extract, Transform, Load) and ELT (Extract, Load, Transform) are processes used to integrate and consolidate data from multiple sources. This section covers a range of open source and commercial tools for performing these tasks, including data ingestion, transformation, and loading into data warehouses or other data storage systems. - Apache NiFi (⭐4.3k) - An automated data integration platform that efficiently moves data between disparate systems. - CloverETL - A comprehensive data integration software suite, ideal for data migration, warehousing, and feeding data into BI and reporting apps. - Hevo - A fully automated, no-code data pipeline platform supporting 150+ integrations across databases, SaaS apps, cloud storage, SDKs, and streaming services. - IBM DataStage - A powerful data integration platform that helps businesses understand, cleanse, transform, and deliver trusted information. - Informatica PowerCenter - A trusted enterprise data integration and management software powering analytics for big data and cloud. - Microsoft SSIS - A reliable platform for building enterprise-level data integration and transformation solutions. - Oracle Data Integrator - A comprehensive data integration platform that covers all data integration requirements, including batch loads, integration processes, and SOA-enabled data services. - Pentaho Data Integration (⭐7.3k) - A user-friendly platform that enables users to ingest, blend, cleanse, and prepare diverse data from any source with visual tools, eliminating the need for coding and complexity. - SAS Data Management - A robust platform that helps transform, integrate, govern, and secure data while improving its overall quality and reliability. - Stitch - A simple yet powerful ETL service that integrates data from dozens of different sources, ideal for businesses of all sizes. - Talend Data Integration - A versatile platform that brings data together from multiple sources to power all your business decisions.
⬆ back to top ### Integration Frameworks The Integration frameworks section includes software tools and libraries that help developers implement and manage integration patterns in their applications. These frameworks are based on well-established Enterprise Integration Patterns (EIPs), which provide a standard vocabulary and architecture for designing and implementing integration solutions. The frameworks in this section can help simplify the process of integrating different systems, applications, and data sources by providing pre-built connectors, message routing, and transformation capabilities. - Apache Camel (⭐5.3k) - A powerful integration framework that uses Enterprise Integration Patterns to connect and unify different transport APIs. - Ballerina (⭐3.5k) - An open-source programming language that simplifies the creation, usage, and combination of network services. - Frank!Framework (⭐110) - A Low-Code Java-based messaging framework to connect your system and application data, that is completely configurable through XML configurations. - Spring Integration (⭐1.5k) - An extension of the Spring programming model that supports Enterprise Integration Patterns, allowing for streamlined integration of various systems.
⬆ back to top ### iPaaS Integration Platform as a Service (iPaaS) is a cloud-based platform that enables businesses to integrate different systems, applications, and data sources with ease. It provides a unified platform for managing data flows between various systems, simplifying the process of integrating different systems and automating workflows. iPaaS tools typically provide a visual interface for designing, deploying, and managing integrations, as well as pre-built connectors and APIs for integrating with popular systems and services. - Anypoint Platform - A powerful integration platform that combines API management and integration capabilities in a single platform, enabling software engineers to integrate various applications with ease. - Boomi AtomSphere - A cloud-native, unified, open, and intelligent platform that connects everything and everyone, allowing software engineers to create and manage integrations easily. - Jitterbit Harmony - A comprehensive integration platform that provides pre-built templates and workflows to automate business processes. It integrates thousands of applications and simplifies integration for software engineers. - IBM Cloud Integration - A next-generation integration platform that uses AI to provide software engineers with an innovative approach to integration. This platform accelerates integration processes, making it faster and more scalable. - Informatica Intelligent Cloud Services - A suite of cloud data management products designed to accelerate productivity and improve speed and scale. Software engineers can use this platform to manage data and integrate applications efficiently. - OpenText Alloy - A powerful enterprise data management platform that empowers organizations to move beyond basic integration and turn data into insights and action. Software engineers can use this platform to manage data and improve business outcomes. - Oracle Integration Cloud Service - A robust platform that accelerates time to go live with pre-built connectivity to any SaaS or on-premises application. Software engineers can use this platform to simplify integration processes and streamline operations. - SnapLogic Intelligent Integration Platform - A comprehensive integration platform that connects various applications and data landscapes. Software engineers can use this platform to integrate data and applications quickly and efficiently. - Software AG webMethods Hybrid Integration Platform - An all-in-one integration platform that enables software engineers to integrate all their applications in a single platform. This platform simplifies integration processes and improves efficiency. - TIBCO Cloud Integration - A flexible platform that enables software engineers to integrate anything with API-led and event-driven integration. This platform empowers everyone to integrate anything, making integration processes faster and more efficient. - Workato - A single platform for integration and workflow automation across your organization, providing software engineers with a powerful platform for simplifying integration processes and streamlining operations.
⬆ back to top ### MaaS Cloud Messaging as a Service (MaaS) refers to cloud-based messaging platforms that enable reliable, secure, and scalable communication between distributed applications and services. These platforms provide various messaging patterns such as publish-subscribe, request-reply, and streaming. They also offer features such as message routing, filtering, transformation, and persistence, and support various protocols and APIs for integration with different systems. - Amazon MQ - A fully managed message broker service that supports Apache ActiveMQ and RabbitMQ. It provides a reliable, secure, and scalable messaging infrastructure for your applications. - Amazon MSK - A fully managed service for Apache Kafka that simplifies the setup, scaling, and management of Kafka clusters. It provides a highly available, durable, and performant messaging service for your data streaming needs. - Amazon SQS - A fully managed message queuing service that enables you to decouple and scale microservices, distributed systems, and serverless applications. It offers a reliable, secure, and highly available message storage and delivery system. - Amazon SNS - A fully managed pub/sub messaging service that enables you to send and receive messages between distributed systems, microservices, and event-driven serverless applications. It offers a flexible, scalable, and cost-effective way to notify subscribers of events and messages. - Alibaba Cloud Message Queue for Apache Kafka - A fully managed Apache Kafka service that provides high-throughput, low-latency, and highly available message streaming for your applications. It offers easy integration with other Alibaba Cloud services and seamless data migration. - Alibaba Cloud Message Queue for RabbitMQ - A distributed and fully managed messaging service that provides high scalability, low latency, and high throughput for your applications. It offers flexible deployment options, easy management, and robust security features. - Alibaba Cloud Message Service - A distributed messaging and notification service that enables concurrent operations and message transfer between applications and decoupled systems. It offers high reliability, scalability, and fault tolerance for your messaging needs. - AlibabaMQ for Apache RocketMQ - A distributed message queue service that supports reliable message-based asynchronous communication among microservices, distributed systems, and serverless applications. It provides high availability, scalability, and durability for your message streaming needs. - Anypoint MQ - An enterprise-class cloud messaging service that is fully integrated with Anypoint Platform. It offers a flexible and reliable messaging infrastructure that enables you to integrate and orchestrate your applications and systems. - Azure Service Bus - A reliable cloud messaging service that offers a variety of messaging patterns, such as pub/sub and request/response, and supports a range of protocols and standards. - CloudAMQP - A RabbitMQ as a Service that provides a fully managed and scalable message broker service for your applications. It offers a simple and easy-to-use web-based management console, advanced monitoring and analytics, and seamless integration with other cloud services. - CloudKarafka - An affordable and straightforward Kafka service that provides fully managed and scalable Kafka clusters on AWS and Google Cloud. It offers a user-friendly web-based dashboard, advanced security features, and seamless integration with other cloud services. - Google Cloud Pub/Sub - A messaging and ingestion service that enables event-driven systems and streaming analytics. It provides high throughput, low latency, and reliable message delivery for your applications. It supports flexible messaging patterns, such as pub/sub and push/pull, and integrates seamlessly with other Google Cloud services. - Huawei Cloud Distributed Message Service - A fully managed and high-performance message queuing service that enables reliable, flexible, and asynchronous communication between distributed applications. It offers high scalability, security, and durability, and supports a variety of messaging patterns and protocols. - Huawei Cloud Distributed Message Service for Kafka - A managed Apache Kafka service that provides easy setup, scaling, and management of Kafka clusters. It offers high availability, performance, and security, and integrates seamlessly with other Huawei Cloud services. - IBM MQ on Cloud - A cloud-based messaging service that enables seamless data transfer across enterprise applications, systems, and services with ease. - IronMQ - An elastic message queue built specifically for the cloud, allowing for reliable and scalable message processing. - Oracle Cloud Streaming - A serverless, real-time event streaming platform with Apache Kafka compatibility, delivering highly performant and cost-effective data processing. - Solace PubSub+ Cloud - An all-in-one event streaming, management, and insights platform, providing complete visibility and control over your data pipeline. - Yandex Message Queue - A cloud messaging service with compatibility to Amazon SQS HTTP API, making it easy to integrate with existing systems and services. - Yandex Managed Service for Apache Kafka - A fully-managed Apache Kafka service with automated scaling, monitoring, and maintenance, providing hassle-free data streaming.
⬆ back to top ### Managed File Transfer Managed File Transfer (MFT) solutions provide secure and reliable file transfer capabilities that help organizations meet regulatory compliance requirements, improve operational efficiency, and reduce the risk of data breaches. MFT software typically includes features such as encryption, digital signatures, user access controls, and detailed audit logs to ensure that data is transmitted securely and can be tracked throughout the transfer process. - ArcESB Managed File Transfer - A comprehensive solution for enterprises seeking to streamline file transfer processes. Its user-friendly interface and powerful capabilities provide an all-in-one MFT solution that can efficiently handle file transfers. - Axway Managed File Transfer - Designed to simplify file transfer services for businesses, this MFT solution offers advanced features and security measures to ensure secure and reliable data transfer. - Titan MFT Server - An enterprise-level MFT server solution with high availability and failover capabilities, providing an efficient way to automate and manage large-scale file transfers. - IBM Sterling Secure File Transfer - A fast and scalable file transfer platform that offers advanced security features and flexible architecture. It is ideal for businesses that require a reliable and secure MFT platform. - Oracle Managed File Transfer - A comprehensive MFT solution that enables secure file exchange and management, providing a reliable and efficient way for businesses to transfer data with ease. - TIBCO Managed File Transfer - A centrally managed MFT platform with powerful automation capabilities and advanced security features, capable of supporting all file transfer use cases. It is an ideal choice for businesses that require a comprehensive MFT solution.
⬆ back to top ### Master Data Management Master Data Management solutions help organizations create a single, authoritative source of accurate and consistent data across different systems, applications, and departments. These tools provide capabilities for data profiling, cleansing, enrichment, and governance, enabling organizations to improve data quality, reduce errors, and increase operational efficiency. - IBM InfoSphere Master Data Management - A comprehensive solution for managing master data that provides a single source of truth across your organization. - Informatica Multidomain MDM - An all-in-one solution that enables you to manage and govern all your master data domains in one place. - Oracle Enterprise Data Management - Helps you adapt and respond to change faster and more effectively by centralizing and managing your master data. - SAP Master Data Governance - Streamlines and improves the quality and consistency of your data by consolidating and centrally governing your master data lifecycle. - SAS MDM - Provides a unified view of your corporate data, integrating information from various sources into one master record for greater accuracy and consistency. - Software AG OneData for MDM - Ensures that only reliable, accurate, and approved information flows through your systems, processes, and applications, providing “one version of the truth.” - Teradata MDM - Establishes an accurate analytical data foundation by using consistent reference data, driving significant ROI. - TIBCO EBX - A single solution to govern, manage, and consume all shared data assets, ensuring greater consistency and accuracy across your organization.
⬆ back to top ### Messaging Message brokers are a type of middleware that allows communication between different applications or systems by facilitating the exchange of messages. They can handle different messaging patterns, such as point-to-point, publish-subscribe, and request-reply, and provide features such as message transformation, routing, and filtering. - Apache ActiveMQ (⭐2.2k) - An open-source message broker that implements Java Message Service (JMS) and allows for synchronous to asynchronous communication conversion. Offers reliable messaging and supports multiple protocols and messaging styles. - Apache Kafka (⭐27k) - A distributed messaging system designed for high-throughput and fault-tolerance. Enables real-time data processing and stream processing applications with its distributed architecture and efficient data storage. - Apache Pulsar (⭐13k) - A distributed pub/sub-messaging system that supports both traditional messaging and streaming use cases. Offers a scalable and flexible architecture and supports multiple messaging protocols. - Apache RocketMQ (⭐20k) - A fast and reliable distributed messaging platform designed for high volume and low latency messaging scenarios. Offers efficient message storage and retrieval and supports various messaging patterns. - Apache Qpid (⭐55) - A messaging tool that supports the Advanced Message Queuing Protocol (AMQP) and multiple programming languages and platforms. Offers enterprise-grade messaging capabilities with reliable and efficient message delivery. - BlazingMQ (⭐2.4k) - A distributed message queueing platform with a focus on efficiency, reliability, and a rich feature set for modern-day workflows. - Centrifugo (⭐7.8k) - An open-source, scalable real-time messaging server that allows you to build interactive applications where events are delivered to online users with minimal delay. - Eclipse Mosquitto (⭐8.3k) - A lightweight open-source message broker that implements the MQTT protocol. Designed for low-power devices and supports message encryption and authentication. - EMQX (⭐13k) - A high-performance, scalable, and reliable open-source MQTT broker for IoT, IIoT, and connected vehicles. - IBM MQ - A robust enterprise-grade messaging system that enables safe and efficient communication between applications. Offers high availability and disaster recovery capabilities and supports multiple messaging protocols. - KubeMQ - A Kubernetes-based message broker and queue system designed for scalability, high availability, and security. Offers seamless integration with Kubernetes and supports various messaging patterns. - NATS (⭐14k) - A lightweight and high-performance messaging system designed for microservices, IoT, and cloud-native systems. Offers efficient and reliable message delivery and supports multiple messaging protocols. - Oracle AQ - A messaging system that provides enterprise message functionality and is widely used within the Oracle database. Offers reliable and efficient message delivery and supports multiple messaging styles. - RabbitMQ (⭐11k) - An open-source message broker that implements AMQP and supports multiple messaging protocols and patterns. Offers reliable and efficient message delivery and supports clustering and high availability. - Redpanda (⭐8.7k) - A streaming platform that is Kafka-compatible and does not require Zookeeper or JVM. Offers high-performance and low-latency messaging and supports stream processing and analytics. - Red Hat AMQ - A messaging system based on open-source communities like Apache ActiveMQ and Apache Kafka. Offers enterprise-grade messaging capabilities with reliable and efficient message delivery. - TIBCO Enterprise Message Service - A standards-based JMS implementation that allows for easy exchange of messages between applications. Offers reliable and efficient message delivery and supports various messaging protocols and patterns. - VerneMQ (⭐3.1k) - A high-performance distributed MQTT message broker designed for scalability and reliability. Offers efficient and reliable message delivery and supports multiple messaging protocols and patterns.
⬆ back to top ### RPA Robotic Process Automation (RPA) solutions are software tools that automate repetitive, rule-based tasks within business processes. RPA bots can perform tasks such as data entry, data extraction, and data processing with accuracy and speed, freeing up human workers to focus on more complex tasks. - Automation Anywhere - An intelligent automation ecosystem that provides Robotic Process Automation (RPA) solutions. This solution is designed to automate business processes and improve operational efficiency while reducing errors and costs. - Blue Prism - A no-code automation RPA platform that offers intelligent automation capabilities for businesses. This platform is designed to reduce operational costs and improve productivity by automating repetitive digital tasks. - UiPath - A platform that provides RPA solutions for automating repetitive digital tasks typically performed by people. This solution offers a comprehensive automation platform that enables businesses to automate their processes and increase efficiency. - WorkFusion - An automation platform for the enterprise that offers RPA solutions. This platform is designed to automate manual tasks, reduce errors and improve efficiency. It offers a comprehensive set of tools for automating workflows and streamlining operations.
⬆ back to top ### Self-Service Integration Self-service and citizen integrator tools are designed to enable non-technical users to build integrations without the need for extensive programming knowledge. These tools typically feature drag-and-drop interfaces and pre-built connectors for popular applications and services.
⬆ back to top ### Workflow engine Software tools that enable the design, execution, and monitoring of complex workflows or business processes. Workflow engines provide a way to automate and streamline business processes, while orchestration engines help to manage the interactions between different systems or services. - Activiti (⭐9.8k) - A lightweight and Java-centric open-source engine for executing BPMN (Business Process Model and Notation) workflows. - Apache Airflow (⭐34k) - A platform that allows you to create, schedule, and monitor workflows programmatically, making it easier to manage and automate complex data pipelines. - Argo Workflows (⭐14k) - An open-source, container-native workflow engine designed to orchestrate parallel jobs on Kubernetes, making it easier to manage and automate complex workflows in a cloud-native environment. - Azkaban (⭐4.3k) - A distributed workflow manager that helps solve the problem of Hadoop job dependencies, making it easier to schedule and manage workflows in large-scale data processing environments. - Bonita (⭐145) - An open-source BPMN engine that comes with a designer and optional development environment, making it easier to build and automate complex business processes. - Cadence (⭐7.7k) - A fault-tolerant, stateful code platform that makes it easier to build and manage complex, long-running applications. - Camunda (⭐3.9k) - An open-source workflow and decision automation platform that helps you automate business processes and decisions in a flexible and scalable way. - Conductor (⭐12k) - A cloud-based orchestration engine designed to run and manage workflows in a microservices architecture. - Elsa Core (⭐5.7k) - A library that enables workflow execution in any .NET Core application, making it easier to automate business processes in .NET environments. - Flowable (⭐7.3k) - A set of open-source business process engines that are compact and efficient, making it easier to automate business processes in a scalable and reliable way. - jBPM (⭐1.5k) - A toolkit for building business applications that helps you automate business processes and decisions, making it easier to manage complex workflows. - StackStorm (⭐5.8k) - A robust automation engine that provides sensors, triggers, rules, workflows, and actions to help you automate and orchestrate complex workflows and applications. - Temporal (⭐9.7k) - An open-source platform that uses a workflow-as-code approach to make your applications more reliable and scalable.
⬆ back to top
## Integration Patterns Integration patterns
provide standardized solutions for common integration problems within an
enterprise. Enterprise Integration Patterns (EIP) provide a common
language for describing integration problems and solutions, while
Integration Architecture Patterns address high-level concerns of
enterprise architects. Service-Oriented Architecture (SOA) patterns
provide guidance for designing and implementing service-oriented
architectures, ensuring that services are scalable, reusable, and
loosely coupled. ### Enterprise Integration Patterns Patterns
from a book by Gregor Hohpe and Bobby Woolf. - Aggregator
- How do we combine the results of individual, but related messages so
that they can be processed as a whole?
- Canonical
Data Model - How can you minimize dependencies when integrating
applications that use different data formats? - Channel
Adapter - How can you connect an application to the messaging system
so that it can send and receive messages? - Channel
Purger - How can you keep ‘left-over’ messages on a channel from
disturbing tests or running systems? - Claim
Check - How can we reduce the data volume of message sent across the
system without sacrificing information content?
- Command
Message - How can messaging be used to invoke a procedure in another
application?
- Competing
Consumers - How can a messaging client process multiple messages
concurrently? - Composed
Message Processor - How can you maintain the overall message flow
when processing a message consisting of multiple elements, each of which
may require different processing?
- Content
Enricher - How do we communicate with another system if the message
originator does not have all the required data items available?
- Content
Filter - How do you simplify dealing with a large message, when you
are interested only in a few data items?
- Content-Based
Router - How do we handle a situation where the implementation of a
single logical function (e.g., inventory check) is spread across
multiple physical systems?
- Control
Bus - How can we effectively administer a messaging system that is
distributed across multiple platforms and a wide geographic area?
- Correlation
Identifier - How does a requestor that has received a reply know
which request this is the reply for? - Datatype
Channel - How can the application send a data item such that the
receiver will know how to process it? - Dead
Letter Channel - What will the messaging system do with a message it
cannot deliver?
- Detour
- How can you route a message through intermediate steps to perform
validation, testing or debugging functions? - Document
Message - How can messaging be used to transfer data between
applications? - Durable
Subscriber - How can a subscriber avoid missing messages while it is
not listening for them? - Dynamic
Router - How can you avoid the dependency of the router on all
possible destinations while maintaining its efficiency? - Envelope
Wrapper - How can existing systems participate in a messaging
exchange that places specific requirements on the message format, such
as message header fields or encryption?
- Event
Message - How can messaging be used to transmit events from one
application to another? - Event-Driven
Consumer - How can an application automatically consume messages as
they become available? - Format
Indicator - How can a
messages data format be designed to allow for possible future changes? - [Guaranteed Delivery](https://www.enterpriseintegrationpatterns.com/patterns/messaging/GuaranteedMessaging.html) - How can the sender make sure that a message will be delivered, even if the messaging system fails? - [Idempotent Receiver](https://www.enterpriseintegrationpatterns.com/patterns/messaging/IdempotentReceiver.html) - How can a message receiver deal with duplicate messages? - [Invalid Message Channel](https://www.enterpriseintegrationpatterns.com/patterns/messaging/InvalidMessageChannel.html) - How can a messaging receiver gracefully handle receiving a message that makes no sense? - [Message](https://www.enterpriseintegrationpatterns.com/patterns/messaging/Message.html) - How can two applications connected by a message channel exchange a piece of information? - [Message Dispatcher](https://www.enterpriseintegrationpatterns.com/patterns/messaging/MessageDispatcher.html) - How can multiple consumers on a single channel coordinate their message processing? - [Message Expiration](https://www.enterpriseintegrationpatterns.com/patterns/messaging/MessageExpiration.html) - How can a sender indicate when a message should be considered stale and thus shouldnt
be processed? - Message
Translator - How can systems using different data formats
communicate with each other using messaging?
- Message
Broker - How can you decouple the destination of a message from the
sender and maintain central control over the flow of messages? - Message
Bus - What is an architecture that enables separate applications to
work together, but in a decoupled fashion such that applications can be
easily added or removed without affecting the others?
- Message
Channel - How does one application communicate with another using
messaging? - Message
Endpoint - How does an application connect to a messaging channel to
send and receive messages? - Message
Filter - How can a component avoid receiving uninteresting messages?
- Message
History - How can we effectively analyze and debug the flow of
messages in a loosely coupled system?
- Message
Router - How can you decouple individual processing steps so that
messages can be passed to different filters depending on a set of
conditions? - Message
Sequence - How can messaging transmit an arbitrarily large amount of
data? - Message
Store - How can we report against message information without
disturbing the loosely coupled and transient nature of a messaging
system?
- Messaging
Bridge - How can multiple messaging systems be connected so that
messages available on one are also available on the others? - Messaging
Gateway - How do you encapsulate access to the messaging system from
the rest of the application? - Messaging
Mapper - How do you move data between domain objects and the
messaging infrastructure while keeping the two independent of each
other? - Normalizer
- How do you process messages that are semantically equivalent, but
arrive in a different format? - Pipes
and Filters - How can we perform complex processing on a message
while maintaining independence and flexibility?
- Point-to-Point
Channel - How can the caller be sure that exactly one receiver will
receive the document or perform the call? - Polling
Consumer - How can an application consume a message when the
application is ready?
- Process
Manager - How do we route a message through multiple processing
steps when the required steps may not be known at design-time and may
not be sequential? - Publish-Subscribe
Channel - How can the sender broadcast an event to all interested
receivers? - Recipient
List - How do we route a message to a list of dynamically specified
recipients? - Request-Reply
- When an application sends a message, how can it get a response from
the receiver? - Resequencer
- How can we get a stream of related but out-of-sequence messages back
into the correct order? - Return
Address - How does a replier know where to send the reply? - Routing
Slip - How do we route a message consecutively through a series of
processing steps when the sequence of steps is not known at design-time
and may vary for each message?
- Scatter-Gather
- How do you maintain the overall message flow when a message needs to
be sent to multiple recipients, each of which may send a reply?
- Selective
Consumer - How can a message consumer select which messages it
wishes to receive?
- Service
Activator - How can an application design a service to be invoked
both via various messaging technologies and via non-messaging
techniques? - Smart
Proxy - How can you track messages on a service that publishes reply
messages to the Return Address specified by the requestor? - Splitter
- How can we process a message if it contains multiple elements, each of
which may have to be processed in a different way?
- Test
Message - What happens, though, if a component is actively
processing messages, but garbles outgoing messages due to an internal
fault?
- Transactional
Client - How can a client control its transactions with the
messaging system?
- Wire
Tap - How do you inspect messages that travel on a point-to-point
channel?
⬆ back to top ### Integration Architecture Patterns Design patterns used to connect systems and applications in an efficient and seamless manner. - API-led Connectivity pattern - Use APIs to connect different systems and applications. - Anti Corruption Layer Pattern - Add a layer to isolate and transform data between systems. - Change Data Capture Pattern - Capture and propagate changes made to a database or data source in real-time. - Hybrid API Management pattern - Manage APIs that span both cloud and on-premises environments using a central control plane. - Hybrid Integration pattern - Integrate systems and applications that are deployed both on-premises and in the cloud using a combination of integration technologies.
⬆ back to top ### Microservice API Patterns MAP (Microservice API Patterns) is a set of proven solutions to common problems encountered when designing, implementing, and maintaining message-based APIs. It focuses on the message representations or payloads exchanged during API calls and their impact on the design and runtime qualities of an API. Proper governance of API specifications and implementations is essential for their long-term maintenance. - Foundation - Frontend Integration - How can client-side end-user interfaces that are physically separated from server-side business logic and data storage be populated and updated with computing results, result sets from searches in data sources, and detailed information about data entities? How can application frontends invoke activities in a backend or upload data to it? - Backend Integration - How can distributed applications and their parts, which have been built independently and are deployed separately, exchange data and trigger mutual activity while preserving system-internal conceptual integrity without introducing undesired coupling? - Public API - How can an API be made available to an unlimited and/or unknown number of API clients outside the organization that are globally, nationally, and/or regionally distributed? - Community API - How can the visibility of and the access to an API be restricted to a closed user group that does not work for a single organizational unit but for multiple legal entities (such as companies, nonprofit/nongovernment organizations, and governments)? - Solution-Internal API - How can access to and usage of an API be limited to an application, for instance, components in the same or another logical layer and/or physical tier? - API Description - Which knowledge should be shared between an API provider and its clients? How should this knowledge be documented?
⬆ back to top ### SOA Patterns SOA patterns are design solutions that provide guidelines and best practices for developing flexible and reusable service-oriented applications. These patterns address various aspects of service-oriented application design, including service identification, interaction, composition, and granularity. - Foundational Inventory Patterns - Canonical Protocol - Defines a common communication protocol between services to enable interoperability and decoupling. - Canonical Schema - Defines a standard data model and format for exchanging data between services. - Domain Inventory - Identifies and categorizes the types of services within a particular domain. - Enterprise Inventory - Identifies and categorizes the types of services within a particular domain. - Logic Centralization - Centralizes business logic within a service layer to reduce redundancy and promote consistency. - Service Layers - Centralizes business logic within a service layer to reduce redundancy and promote consistency. - Service Normalization - Centralizes business logic within a service layer to reduce redundancy and promote consistency.
⬆ back to top ## Resources Some useful specifications, standards, articles and documentation. ### API Specification - API Blueprint (⭐8.6k) - A powerful high-level language for designing and documenting APIs that allows software engineers to easily collaborate and create efficient APIs. - AsyncAPI (⭐3.8k) - An essential tool for developing Event-Driven Architectures (EDA) and enables engineers to build a better tooling ecosystem. - GraphQL (⭐14k) - A sophisticated query language and runtime for building efficient APIs that empowers engineers to retrieve data from existing systems with ease. - JSON:API (⭐7.3k) - A standardized specification for building APIs that simplifies the representation of resources, relationships, and metadata, making it easier for software engineers to create efficient APIs. - OpenAPI (ex.Swagger) (⭐28k) - A language-agnostic specification for creating RESTful APIs that enables both humans and machines to understand the capabilities of a service without the need for source code or documentation. - RAML (⭐3.8k) - A RESTful API Modeling Language that allows software engineers to design and create efficient APIs by modeling resources, endpoints, and interactions. - WSDL - A powerful XML-based interface description language for SOAP-based services that enables software engineers to describe the functionality of a web service and automate the creation of client code.
⬆ back to top ### Articles - API-Security-Checklist (⭐21k) - Best practices about REST API security. - Architectural Styles and the Design of Network-based Software Architectures - Roy Fielding’s dissertation defining REST. - Enterprise Integration Using REST - Discusses the constraints and flexibility that you have with nonpublic APIs, and lessons learned from doing large scale RESTful integration across multiple teams. - Richardson Maturity Model - Explained by Martin Fowler, originally presented by Leonard Richardson. - Web API Design: Crafting interfaces that developers love - Creating Web APIs that are consistent, intuitive, and user-friendly for developers.
⬆ back to top ### Certifications⬆ back to top ### Data Formats - Apache Avro (⭐2.7k) - Data serialization system that provides compact, fast, and efficient serialization of structured data. It supports schema evolution, allows for efficient data compression, and is designed to work well with big data processing frameworks. - BSON - Binary-encoded serialization format for JSON-like documents that is designed to be lightweight and efficient. It supports rich data types and is widely used in NoSQL databases, such as MongoDB. - CSV - A simple and widely used data format that stores tabular data in plain text. It is easy to read and write, and can be processed by most programming languages. - JSON - A lightweight and easy-to-read data format that is widely used for data exchange. It supports a wide range of data types and is compatible with many programming languages. - NDJSON (⭐632) - A standard for delimiting JSON objects in stream protocols. It allows for efficient processing of large JSON datasets and is widely used in big data processing. - Protocol Buffers (⭐63k) - A language-neutral and platform-neutral serialization mechanism that is designed to be highly efficient and extensible. It supports rich data types and is widely used in distributed systems, such as gRPC and Apache Kafka. - XML - A flexible and widely used markup language that is used for storing and exchanging structured data. It supports rich data types and is compatible with many programming languages. - YAML (⭐309) - A human-friendly and easy-to-read data serialization format that is widely used for configuration files and data exchange. It supports rich data types and is compatible with most programming languages.
⬆ back to top ### Integration Styles - File Transfer - Data is exchanged through file exchange between applications. - Messaging - Applications exchange messages using a messaging infrastructure. - Remote Procedure Invocation - Applications call functions or procedures on a remote server over a network. - Shared Database - Multiple applications access and manipulate the same data through a common database.
⬆ back to top ### Market Analysis - API Management - Gartner Critical Capabilities for Full Life Cycle API Management - Gartner Magic Quadrant for Full Life Cycle API Management - The Forrester Wave: API Management Solutions - BPM - Gartner Critical Capabilities for Intelligent Business Process Management Suites - Gartner Magic Quadrant for Intelligent Business Process Management Suites - ETL - Gartner Critical Capabilities for Data Integration Tools - Gartner Magic Quadrant for Data Integration Tools - iPaaS - Gartner Critical Capabilities for Enterprise Integration Platform as a Service - Gartner Magic Quadrant for Enterprise Integration Platform as a Service - The Forrester Wave: Enterprise iPaaS - RPA - Gartner Critical Capabilities for Robotic Process Automation - Gartner Magic Quadrant for Robotic Process Automation - The Forrester Wave: Robotic Process Automation
⬆ back to top ### Protocols - AMQP 0-9-1 - A message queuing protocol that enables the exchange of messages between applications or systems. AMQP 0-9-1 provides reliability, security, and flexibility to handle complex messaging scenarios. - AMQP 1.0 - A widely accepted message queuing protocol that provides reliable, interoperable, and efficient messaging between systems. AMQP 1.0 supports a broad range of messaging scenarios and is ideal for complex enterprise-level applications. - CoAP - A specialized application protocol designed for constrained devices in the Internet of Things (IoT) ecosystem. CoAP offers a lightweight, low-overhead communication mechanism to support resource-constrained devices. - HTTP - A widely used protocol that facilitates communication between web servers and clients. HTTP enables the transfer of data over the internet, making it possible for web applications to function seamlessly. - JSON-RPC - A simple and lightweight remote procedure call protocol that enables communication between systems using JSON data. JSON-RPC offers stateless communication and is suitable for resource-constrained devices. - MQTT - A lightweight and efficient publish-subscribe protocol that supports messaging between devices. MQTT enables low-overhead communication and is ideal for IoT and mobile applications. - OpenMessaging (⭐286) - A cloud native, vendor-neutral open specification for distributed messaging. - SOAP - A messaging protocol that uses XML to enable communication between systems. SOAP supports a wide range of messaging scenarios, including distributed computing and enterprise applications. - STOMP - A messaging protocol that offers simple, text-oriented communication between systems. STOMP is ideal for low-latency, high-performance messaging scenarios.
⬆ back to top ### Standard APIs - JDBC - A Java-based API that provides universal data access to relational databases. JDBC offers a consistent and efficient way to access and manipulate data, making it a popular choice among developers. - JMS - A messaging API that enables Java applications to send and receive messages. JMS supports reliable messaging and is widely used in enterprise-level applications. - ODBC - A widely accepted API that provides a standardized way to access data from various database management systems. ODBC offers a consistent interface to access data, making it easy to develop database applications. - OData - An open protocol that enables the creation and consumption of queryable and interoperable REST APIs. OData simplifies the development of REST APIs and offers a standardized way to access data. - Standard Webhooks (⭐1k) - Open source tools and guidelines for sending webhooks easily, securely, and reliably.
⬆ back to top ### Structure and Validation - JSON Schema (⭐3.1k) - A powerful tool for validating the structure of JSON data. JSON Schema enables developers to ensure that JSON data conforms to a specific structure, making it easier to process and manipulate. - Schematron - A rule-based validation language that enables developers to define and validate business rules, data reporting, quality control, and other validation scenarios. Schematron offers a flexible way to validate XML documents. - XML Schema - A schema language that offers facilities for describing the structure and constraining the contents of XML documents. XML Schema enables developers to ensure that XML data conforms to a specific structure, making it easier to process and manipulate.
⬆ back to top ## Contributing Your contributions are always welcome! Please take a look at the contribution guidelines first.