模拟Web事件
本文介绍了一个Python包，用于生成半随机的Web事件，这些事件遵循预定义的约束，使其看起来更真实。通过使用此包，您可以为数据分析工程师职位构建投资组合，演示创建Web会话、衡量转化率以及按广告系列或来源划分指标的能力。
安卓模拟器 模拟滑动事件
You want to implement a new streaming pipeline on your workplace and need to show your managers a Proof Of Concept. This POC should allow you to demonstrate some of the functionalities, in this case, generate real-time metrics. However, there is a limitation: you can’t use production data on a POC. How do you solve that?
您想要在工作场所实施新的流传输管道，并且需要向您的经理展示概念证明。 此POC应该使您能够演示一些功能，在这种情况下，可以生成实时指标。 但是，有一个限制：您不能在POC上使用生产数据。 您如何解决？
If your answer was to generate fake events, then you are right. It will probably be the best solution. Then you create some random events and push then to the pipeline. You’ll soon realize that random is not the same as fake. You certainly don’t want to show a chart like this to your managers:
如果您的答案是生成假事件，那么您是对的。 这可能是最好的解决方案。 然后创建一些随机事件，然后推送到管道。 您很快就会意识到，随机与伪造并不相同。 您当然不希望向经理显示这样的图表：
Random events don’t make a point; they don’t sell anything. Let’s have a grasp at another metric created on top of random events:
随机事件没有意义； 他们什么都不卖。 让我们了解在随机事件之上创建的另一个指标：
Now let’s say you want to build your portfolio for an Analytics Engineer position. You possibly need to show that you are capable of creating web sessions, measuring conversion rate and splitting metrics by campaign or source. How do you build your damn portfolio if you don’t have data? And again, generating random data won't allow you to measure anything.
现在，假设您要为Analytics(分析)工程师职位建立投资组合。 您可能需要证明自己具有创建网络会话，衡量转化率以及按广告系列或来源划分指标的能力。 如果没有数据，如何建立该死的投资组合？ 再说一次，生成随机数据将使您无法进行任何测量。
创建半随机事件 (Creating semi-random events)
Thinking about those problems, I decided to create a python package to generate fake events that are not entirely random. They follow a set of constraints and allow you to create beautiful things with fake data that looks like real. Look again at the page-views over time chart:
考虑到这些问题，我决定创建一个python包来生成并非完全随机的虚假事件。 它们遵循一系列约束，并允许您使用看起来像真实的假数据来创建漂亮的东西。 再次查看时间图表内的浏览量：
If we look at the page-views funnel, we would expect to see more visits on Home and much less on payment and confirmation, right? Let’s check if that is true.
如果我们查看综合浏览量渠道，那么我们期望看到更多的“首页”访问，而更少的付款和确认访问，对吗？ 让我们检查一下是否正确。
如何自行创建虚假的网络事件？ (How to create fake web events on your own?)
Is quite simple! The package is available on PyPI and source code on GitHub. To install you do pip install, as you would for any other package:
很简单！ 该软件包可在PyPI上获得 ， 源代码可在GitHub上获得 。 要进行安装 ，请像其他任何软件包一样进行pip install ：
pip install fake_web_eventsIt is also simple to use. See the example below that will print events to stdout.
使用也很简单。 请参见下面的示例，该示例会将事件打印到stdout。
from fake_web_events import Simulationsimulation = Simulation(user_pool_size=100, sessions_per_day=10000)
events = simulation.run(duration_seconds=10)for event in events:
print(event)You can do anything you want inside the loop. Send it to Kafka or Kinesis. Process it with Spark streaming and create some metrics. Save to a Postgres and build some dashboards using Metabase. Show off your SQL skills on your portfolio. Or get that POC approved.
您可以在循环内做任何您想做的事情。 将其发送到Kafka或Kinesis。 使用Spark流处理它并创建一些指标。 保存到Postgres并使用元数据库构建一些仪表板。 在您的产品组合上炫耀您SQL技能。 或获得该POC批准。
An event created by this package will look like this:
此包创建的事件将如下所示：
{
"event_timestamp": "2020-07-05 14:32:45.407110",
"event_type": "pageview",
"page_url": "http://www.dummywebsite.com/home",
"page_url_path": "/home",
"referer_url": "www.instagram.com",
"referer_url_scheme": "http",
"referer_url_port": "80",
"referer_medium": "internal",
"utm_medium": "organic",
"utm_source": "instagram",
"utm_content": "ad_2",
"utm_campaign": "campaign_2",
"click_id": "b6b1a8ad-88ca-4fc7-b269-6c9efbbdad55",
"geo_latitude": "41.75338",
"geo_longitude": "-86.11084",
"geo_country": "US",
"geo_timezone": "America/Indiana/Indianapolis",
"geo_region_name": "Granger",
"ip_address": "209.139.207.244",
"browser_name": "Firefox",
"browser_user_agent": "Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_5_5; rv:1.9.6.20) Gecko/2012-06-06 09:24:19 Firefox/3.6.20",
"browser_language": "tn_ZA",
"os": "Android 2.0.1",
"os_name": "Android",
"os_timezone": "America/Indiana/Indianapolis",
"device_type": "Mobile",
"device_is_mobile": true,
"user_custom_id": "vsnyder@hotmail.com",
"user_domain_id": "3d648067-9088-4d7e-ad32-45d009e8246a"
}Fake Web Events is using Faker to create some data; other fields are randomly generated based on predefined weights (that you can customize in a YAML file).
Fake Web Events正在使用Faker创建一些数据。 其他字段是根据预定义的权重(您可以在YAML文件中自定义)随机生成的。
虚假的网络事件 (Fake web events under the hood)
I assume you might be interested in understanding how those events are created and in what is the logic behind it. I can tell you straight away that it’s not Alien technology :).
我认为您可能对了解如何创建这些事件以及其背后的逻辑感兴趣。 我可以马上告诉您，这不是外星人技术:)。
用户数 (Users)
We start by creating a pool of users. Some fields are entirely random (coming from Faker), others follow a set of predefined weights. For example, the user will get a browser assigned using the following weights:
我们首先创建一个用户池。 一些字段是完全随机的(来自Faker)，其他字段则遵循一组预定义的权重。 例如，用户将获得使用以下权重分配的浏览器：
browsers:
Chrome: 0.5
Firefox: 0.25
InternetExplorer: 0.05
Safari: 0.1
Opera: 0.1Meaning that Chrome will appear on roughly 50% of users and Firefox on 25%.
这意味着Chrome将在大约50％的用户中显示，而Firefox在25％的用户中显示。
A user has essentially the same JSON structure than an event, except by the fact that it doesn’t contain timestamps or page information. When we run the simulation, users will be randomly picked from the pool to start “navigating the website”. Users might return to the site later, that’s why they are chosen with reposition.
用户具有与事件基本相同的JSON结构，但事实是它不包含时间戳或页面信息。 当我们运行模拟时，将从池中随机挑选用户以开始“浏览网站”。 用户可能稍后会返回该站点，这就是为什么选择重新定位他们的原因。
大事记 (Events)
To create events, we simulate sessions. There are a few “pages” created in the simulation. The user will start either at Home, Product A or Product B pages. At each iteration, the user could stay on the same page, move to another page, or finalize the session. Below are the probabilities for a user that is currently on Home:
为了创建事件，我们模拟会话。 在模拟中创建了一些“页面”。 用户将从首页，产品A或产品B页面开始。 在每次迭代时，用户可以停留在同一页面上，移至另一页面或完成会话。 以下是当前在家中的用户的概率：
home:
home: 0.45
product_a: 0.17
product_b: 0.12
session_end: 0.26The simulation tracks all users and generates an event every time it reaches a new page. Once the user reaches Session End, his session is dropped. Follows a diagram of the possible user events during the simulation:
模拟会跟踪所有用户，并在每次到达新页面时生成一个事件。 用户到达会话结束后，其会话将被丢弃。 遵循模拟过程中可能的用户事件的图表：
模拟 (Simulation)
The simulation will run in batches, and its internal clock will usually go much faster than the standard time. The exception is if you are running it with very large user_pool_size and sessions_per_day.
模拟将分批运行，其内部时钟通常会比标准时间快得多。 唯一的例外是，如果你是非常大的user_pool_size和sessions_per_day运行它。
The simulation will create a generator object. It won’t run or use resources until you start iterating over it.
模拟将创建一个生成器对象。 在开始对其进行迭代之前，它不会运行或使用资源。
接下来是什么？ (What comes next?)
I do have a few ideas on what to do next on this project. Some are simple bug fixes or improved implementation of how events are generated and tracked. But there is one broad idea that I would like to explore more:
对于这个项目的下一步我确实有一些想法。 有些是简单的错误修复，或者是事件生成和跟踪的改进实现。 但是有一个广泛的想法，我想探索更多：
Producing microservices events on top of web events.
在Web事件之上产生微服务事件。
My idea is to add another module to the simulation that would create different events based on the microservices architecture. I can think of events like this:
我的想法是向模拟添加另一个模块，该模块将基于微服务架构创建不同的事件。 我可以想到这样的事件：
- user_created user_created
- order_created order_created
- payment_completed 付款完成
- payment_failed 支付失败
- order_dispatched order_Dispatched
- order_completed order_completed
Microservices events would be useful to create fake transactional databases that are bonded to web events. I think it would be useful for a lot of people as it would allow even more use cases.
微服务事件对于创建绑定到Web事件的虚假事务数据库很有用。 我认为这将对很多人有用，因为它将允许更多用例。
If you like this project, please show your love by starring it on GitHub.
如果您喜欢这个项目，请在GitHub上加注您的爱意。
Everyone can also contribute: create some issues, feature requests, or even Pull Requests if you are comfortable in touching the code!
每个人都可以做出贡献 ：创建一些问题，功能请求，甚至是拉动请求(如果您习惯于触摸代码)！
翻译自: https://towardsdatascience.com/simulating-web-events-7199bf8afcfd
安卓模拟器 模拟滑动事件
扫一扫
1万+
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
