Introducing Hosting Based Interfacing

HBI(Hosting-Based Interfacing)是一种用于提高分布式应用程序间互操作性的架构,它通过让软件组件在其本地环境中执行来自对等方的逻辑来实现资源的高效利用和事务的透明重试。与传统的同步访问远程资源的方法相比,HBI能够显著提升性能并减少因同步执行和分布式数据缓存带来的问题。
[b]HBI - Hosting Based Interfacing [/b]


[b]What is (not) HBI? [/b]

[quote]HBI is about interoperability among distributed applications (software components), built with same or different technologies. As the name implies, HBI is an architecture that software components communicate with each others by hosting the execution of logics from peers. HBI is to get rid of synchronous access to remote resources, make all code executed natively and locally (in both concept and practice) whereas asynchronously. So transactional logics can be naturally grouped into atomic units and performed in parallel ways. On optimistic commit failures, transactions are possible to be retried automatically and transparently. At the same time performance drop and pitfalls due to synchronous execution and distributed data cache are get rid of, as well.

SQL is essentially semi-HBI: There is an interesting fact that wherever you wrote your SQL code, it is executed in the server environment, and you know your SQL always access local resources like tables, views and stored-procedures etc. SQL transactions naturally consist of arbitrary statements between two COMMITs. Pure SQL transactions are possible to be transparently retried on optimistic commit failures.

On the contrary is IBI - Invocation Based Interfacing. Those paradigms try to hide resources behind some mechanisms so that access to resources are wrapped/encapsulated in synchronous ways, regardless they are local or remote. Some define service invocation interfaces, like RMI (including EJB), Web-Service, where remote routines encapsulate their resources and are wrapped like client-local code. Some allow data resources distributed to client environment and wrapped with client-local library code that transparently make data available, handle validity, and manage consistency, like various OODBs, ORM. Transactions can rarely be transparently retried on these infrastructures. [/quote]


[b]What does HBI cost? [/b]

[quote]Distribution and concurrency/parallel become explicit, asynchronous complexity instead of implicit, synchronous complexity.

Synchronous programming habit will be broken, programmers will have to consciously know that some blocks of their code will run asynchronously at environments (domains) other than the local application they are developing. Code relies on the result of some remote manipulations can not be programmed as a subsequent block of that manipulation code, but must be implemented as local task agent objects that the remote peer can send back to indicate the result.

Higher security profile. Applications need to enforce sandboxes to host executions of objects from other domains, against malfunctioning peers and hostile attacks. [/quote]


[b]What HBI benefits? [/b]

[quote]The architecture of large RAM/storage shared by many CPUs/cores is best benefited. Large data sets in a single memory space and local storage are shared as-is to all domains requiring them, manipulations to these resources are hosted by local CPUs/cores having direct access to them. This is the most efficient way for distributed domain resource sharing.

As a result, performance is a biggest hit. All data and logics become local and native, and can be greatly optimized with explicit parallel (parallel candidates are expressed by application code as a side effect). No data replicated anymore, large amount of redundant memory and bandwidth saved.

Much less pitfalls due to distribution of objects. Neither objects are distributed anymore, nor do they need to be transfered over wire, objects now always identify themselves at where they actually are, no shadow copies or stubs or proxies used to create subtle errors and harden debugging.

Server applications are much more stable. Most new client requirements those used to need service API changes in traditional approaches now can be unilaterally achieved by modifying only the client application.

Client applications can go much richer without resistance from the server side, do not need to change/restart server applications (i.e. less permissions/efforts required) in order to evolve client applications.

Client applications have their own rich domain models that focus on their own functionalities, much better separated from functionalities like persistence and work flows those server applications need to care. Client applications can focus on their own domain model which is regarded by server applications by translating server domain model changes to client behaviors through delivery of task agent life scripts.

More effective Object Orientation. No data packet based protocol anymore, all logics become behaviors of either domain objects or agent objects triggered by task agent life script.

P2P communications become more natural. Clients are also domains, they are at the same priority of server components in inter-domain communications.

Much more realtime-interactive features can be naturally implemented to enhance user experience. Built-in server-push and P2P-push ability well enable it. [/quote]
Spring XD 是一个开源的分布式数据处理平台,主要用于构建数据管道和集成任务。它提供了一种简单而高效的方式来处理实时数据流和批量数据。 ### 主要特点: 1. **模块化设计**:Spring XD 采用模块化架构,支持多种数据源、处理器和存储系统,使得用户可以灵活地组合不同的模块来满足特定的需求。 2. **分布式处理**:通过使用Apache ZooKeeper进行协调,Spring XD 能够在多个节点上分布式运行任务,从而提高系统的可扩展性和可靠性。 3. **丰富的模块库**:Spring XD 提供了大量的预定义模块,包括数据源(如Kafka、RabbitMQ)、处理器(如转换、过滤)和存储系统(如HDFS、HBase)。用户也可以自定义模块来扩展功能。 4. **脚本支持**:支持使用JavaScript、Python等脚本语言编写处理器逻辑,使得开发更加灵活和快速。 5. **易于部署和管理**:Spring XD 提供了命令行工具和Web界面,方便用户进行部署、监控和管理。 6. **与Spring生态系统集成**:作为Spring家族的一部分,Spring XD 可以无缝集成到现有的Spring应用程序中,利用Spring框架的强大功能。 ### 应用场景: - **实时数据处理**:适用于需要对流式数据进行实时处理的场景,如日志分析、事件驱动的应用等。 - **ETL流程**:用于提取、转换和加载数据,特别适合大数据环境下的数据集成任务。 - **微服务架构**:在微服务架构中,Spring XD 可以用来处理服务间的消息传递和数据流。 ### 安装步骤: 1. **下载并解压安装包**。 2. **配置环境变量**,确保Java和Maven已正确安装。 3. **启动ZooKeeper**,因为Spring XD依赖于ZooKeeper进行集群管理。 4. **运行Spring XD单节点模式**,可以使用`xd-singlenode`脚本启动。 5. **访问Web界面**,通常位于`http://localhost:8080/`,在这里可以进行模块的创建和管理。 ### 示例代码: 以下是一个简单的例子,展示了如何使用Spring XD创建一个数据管道,该管道从Kafka读取消息,将其转换为大写,然后输出到控制台。 ```shell # 创建Kafka源模块 module create --name kafka-source --type source --definition "kafka --brokers=localhost:9092 --topic=input" # 创建转换模块 module create --name uppercase-transform --type processor --definition "log --expression='payload.toUpperCase()'" # 创建控制台输出模块 module create --name console-sink --type sink --definition "log" # 创建数据管道 stream create --name uppercase-pipeline --definition "kafka-source > uppercase-transform > console-sink" # 启动数据管道 stream deploy --name uppercase-pipeline ```
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值