Kafka+Storm+ElasticSearch

最新推荐文章于 2020-09-16 01:06:18 发布

转载最新推荐文章于 2020-09-16 01:06:18 发布 · 2k 阅读

big data 专栏收录该内容

42 篇文章

订阅专栏

Loggly通过采用最新的一系列社交媒体技术作为多租户、地理分布式的实时日志管理系统的核心组件进行案例研究。介绍了如何在AWS服务上构建第二代系统，包括Amazon Route53 DNS、多区域VPC、弹性负载均衡、Amazon RDS等，以实现计算和索引容量的灵活扩展。

Infrastructure at Scale: Apache Kafka, Twitter Storm & Elastic Search

Published on Nov 29, 2013

This is a technical architect's case study of how Loggly has employed the latest social-media-scale technologies as the backbone ingestion processing for our multi-tenant, geo-distributed, and real-time log management system. This presentation describes design details of how we built a second-generation system fully leveraging AWS services including Amazon Route 53 DNS with heartbeat and latency-based routing, multi-region VPCs, Elastic Load Balancing, Amazon Relational Database Service, and a number of pro-active and re-active approaches to scaling computational and indexing capacity.

The talk includes lessons learned in our first generation release, validated by thousands of customers; speed bumps and the mistakes we made along the way; various data models and architectures previously considered; and success at scale: speeds, feeds, and an unmeltable log processing engine.

https://www.youtube.com/embed/LpNbjXFPyZ0