python自动生成表格,动态生成表格时,如何使用Python BeautifulSoup来获取表格信息?...

博主尝试从Bloomberg网站获取收益日历信息,发现其依赖jQuery处理日期选择事件。通过浏览器开发者工具,他们观察到与日期选择相关的jQuery代码,并希望了解如何利用Requests库发送自定义日期,使页面刷新并显示对应日期的收益数据。解决方案建议通过分析网络请求,找到更改日期参数的API,直接传递自定义日期来获取所需信息。

I am trying to get earnings calendar information without using yahoo or nasdaq since they are not always accurate.

Bloomberg's website has earnings information at the following url:

https://www.bloomberg.com/markets/earnings-calendar/us

When I inspect the html source for the page, I found the html code block that controls the date picking calendar tool and inspected the some of the event code that I've marked EV#1, EV#2, EV#3, and EV#4 for reference.

button

I've found that each of these EV#n events refer to some jQuery code that handle click events.

When I highlight the event related toolbox in the web inspector at each EV#n, I see code descriptions with either JQuery or Bubbling, DOM2.

I have no idea what these mean, but perhaps it's not so important at this stage. I've included the descriptions and which EV#'s correspond to the code below. Perhaps it is helpful..

JQuery: (EV#1)

tellMeTheSelectedDate: function() {

r.locale(this.locale);

var e = this.$datePicker().datepicker("getDate"),

t = r(e).format("L");

a.log(t)

}

Bubbling, DOM2: (EV#1, EV#2, EV#3, EV#4)

function An(e) {

if (Gi) {

var t = Cn(e);

if (t && this.id == t.id) {

var n = In(t);

if (n && kn(e)) {

var r = Rn(n);

r.PX263 = d(t), re("PX217", r), ji++, Fi <= ji && (Gi = !1, Dn(!1))

}

}

}

}

Bubbling, DOM2: (EV#1, EV#4)

u = b.handle = function(e) {

return typeof d === R || e && d.event.triggered === e.type ? void

0 : d.event.dispatch.apply(u.elem, arguments)

}

jQuery: (EV#4)

function() {

return e.datepicker._datepickerShowing &&

e.datepicker._lastInput === t[0] ? e.datepicker._hideDatepicker()

: e.datepicker._datepickerShowing && e.datepicker._lastInput !==

t[0] ? (e.datepicker._hideDatepicker(),

e.datepicker._showDatepicker(t[0])) :

e.datepicker._showDatepicker(t[0]), !1

}

GOAL:

I have no experience with JQuery, but I know it uses CSS descriptors to dynamically alter web content.

Is it possible for me to use Requests and send a "picked date" (a date string) to the date picking widget by modifying whatever variable stores this value from the calendar tool so that the page refreshes(is this right?) and displays earnings information in the table based on the date I have passed via script?

Would I need to emulate a "click" event to get the page to update with my date?

Getting the data from the table using bs4 is no issue, but my main problem is the date selection.

Thanks in advance

解决方案

If you open developer tools in your browser (F12) and go to the network tab, you can sniff out the request the page is making when you change the date on the calendar. In doing so, you can pass in whichever date you like as a URL parameter:

https://www.bloomberg.com/markets/api/calendar/earnings/US?locale=en&date=2018-04-30

以下是一个简单的python程序,用于爬取51job网站上的招聘信息,并将其存储为表格形式。本程序使用BeautifulSoup和pandas库。 ```python import requests from bs4 import BeautifulSoup import pandas as pd # 定义要爬取的页面链接 url = 'https://search.51job.com/list/000000,000000,0000,00,9,99,python,2,1.html' # 发送HTTP请求,获取页面内容 response = requests.get(url) # 使用BeautifulSoup解析页面内容 soup = BeautifulSoup(response.content, 'html.parser') # 查找所有class为"t1"的a标签 job_titles = soup.find_all('a', class_='t1') # 查找所有class为"t2"的span标签 company_names = soup.find_all('span', class_='t2') # 查找所有class为"t3"的span标签 locations = soup.find_all('span', class_='t3') # 查找所有class为"t4"的span标签 salaries = soup.find_all('span', class_='t4') # 创建一个空的DataFrame对象 df = pd.DataFrame(columns=['Job Title', 'Company Name', 'Location', 'Salary']) # 将招聘信息添加到DataFrame对象中 for i in range(len(job_titles)): job_title = job_titles[i].get_text().strip() company_name = company_names[i].get_text().strip() location = locations[i].get_text().strip() salary = salaries[i].get_text().strip() df.loc[len(df)] = [job_title, company_name, location, salary] # 将DataFrame对象保存为CSV文件 df.to_csv('job_information.csv', index=False) print('Job information has been saved to job_information.csv.') ``` 运行完毕后,程序将在当前目录下生成一个名为job_information.csv的文件,其中包含了爬取到的招聘信息。 注意:本程序仅供学习和参考,如有商业用途请遵守相关法律法规。
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符  | 博主筛选后可见
 
红包 添加红包
表情包 插入表情
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值