fix协议介绍2-登入消息

本教程提供了一系列关于Kubernetes(K8s)的部署方法及资源管理配置的视频课程链接,涵盖从开发测试环境到生产环境的各种部署方案,并详细介绍了K8s中重要资源的概念和使用方法。

发布一个k8s部署视频:https://edu.youkuaiyun.com/course/detail/26967

课程内容:各种k8s部署方式。包括minikube部署,kubeadm部署,kubeasz部署,rancher部署,k3s部署。包括开发测试环境部署k8s,和生产环境部署k8s。

腾讯课堂连接地址https://ke.qq.com/course/478827?taid=4373109931462251&tuin=ba64518

第二个视频发布  https://edu.youkuaiyun.com/course/detail/27109

腾讯课堂连接地址https://ke.qq.com/course/484107?tuin=ba64518

介绍主要的k8s资源的使用配置和命令。包括configmap,pod,service,replicaset,namespace,deployment,daemonset,ingress,pv,pvc,sc,role,rolebinding,clusterrole,clusterrolebinding,secret,serviceaccount,statefulset,job,cronjob,podDisruptionbudget,podSecurityPolicy,networkPolicy,resourceQuota,limitrange,endpoint,event,conponentstatus,node,apiservice,controllerRevision等。

第三个视频发布:https://edu.youkuaiyun.com/course/detail/27574

详细介绍helm命令,学习helm chart语法,编写helm chart。深入分析各项目源码,学习编写helm插件


————————————————

Logon消息结构:

 

ield or ComponentField NameFIXML nameReq'dCommentsDepr.
dummy.gifuploading.4e448015.gif转存失败重新上传取消ComponentStandardHeaderBaseHeadercheckmark.gifuploading.4e448015.gif转存失败重新上传取消

MsgType = A

 
dummy.gifuploading.4e448015.gif转存失败重新上传取消98EncryptMethod checkmark.gifuploading.4e448015.gif转存失败重新上传取消

(Always unencrypted)

 
dummy.gifuploading.4e448015.gif转存失败重新上传取消108HeartBtInt checkmark.gifuploading.4e448015.gif转存失败重新上传取消

Note same value used by both sides

 
dummy.gifuploading.4e448015.gif转存失败重新上传取消95RawDataLength@RawDataLength 

Required for some authentication methods

 
dummy.gifuploading.4e448015.gif转存失败重新上传取消96RawData@RawData 

Required for some authentication methods

 
dummy.gifuploading.4e448015.gif转存失败重新上传取消141ResetSeqNumFlag  

Indicates both sides of a FIX session should reset sequence numbers

 
dummy.gifuploading.4e448015.gif转存失败重新上传取消789NextExpectedMsgSeqNum  

Optional, alternative via counterparty bi-lateral agreement message gap detection and recovery approach (see "Logon Message NextExpectedMsgSeqNum Processing" section)

 
dummy.gifuploading.4e448015.gif转存失败重新上传取消383MaxMessageSize  

Can be used to specify the maximum number of bytes supported for messages received

 
dummy.gifuploading.4e448015.gif转存失败重新上传取消ComponentMsgTypeGrpMsgTypeGrp   
dummy.gifuploading.4e448015.gif转存失败重新上传取消464TestMessageIndicator  

Can be used to specify that this FIX session will be sending and receiving "test" vs. "production" messages.

 
dummy.gifuploading.4e448015.gif转存失败重新上传取消553Username@Username   
dummy.gifuploading.4e448015.gif转存失败重新上传取消554Password@Password 

Note: minimal security exists without transport-level encryption.

 
dummy.gifuploading.4e448015.gif转存失败重新上传取消925NewPassword@NewPassword 

Specifies a new password for the FIX Logon. The new password is used for subsequent logons.

 
dummy.gifuploading.4e448015.gif转存失败重新上传取消1400EncryptedPasswordMethod@EncPwdMethod   
dummy.gifuploading.4e448015.gif转存失败重新上传取消1401EncryptedPasswordLen    
dummy.gifuploading.4e448015.gif转存失败重新上传取消1402EncryptedPassword@EncPwd   
dummy.gifuploading.4e448015.gif转存失败重新上传取消1403EncryptedNewPasswordLen    
dummy.gifuploading.4e448015.gif转存失败重新上传取消1404EncryptedNewPassword@EncNewPwd 

Encrypted new password- encrypted via the method specified in the field EncryptedPasswordMethod(1400)

 
dummy.gifuploading.4e448015.gif转存失败重新上传取消1409SessionStatus@SessStat 

Session status at time of logon. Field is intended to be used when the logon is sent as an acknowledgement from acceptor of the FIX session.

 
dummy.gifuploading.4e448015.gif转存失败重新上传取消1137DefaultApplVerID@DefApplVerIDcheckmark.gifuploading.4e448015.gif转存失败重新上传取消

The default version of FIX messages used in this session.

 
dummy.gifuploading.4e448015.gif转存失败重新上传取消1407DefaultApplExtID@DfltApplExtID 

The default extension pack for FIX messages used in this session

 
dummy.gifuploading.4e448015.gif转存失败重新上传取消1408DefaultCstmApplVerID  

The default custom application version (dictionary) for FIX messages used in this session

 
dummy.gifuploading.4e448015.gif转存失败重新上传取消58Text@Txt 

Available to provide a response to logon when used as a logon acknowledgement from acceptor back to the logon initiator.

 
dummy.gifuploading.4e448015.gif转存失败重新上传取消354EncodedTextLen@EncTxtLen 

Must be set if EncodedText field is specified and must immediately precede it.

 
dummy.gifuploading.4e448015.gif转存失败重新上传取消355EncodedText@EncTxt 

Encoded (non-ASCII characters) representation of the Text field in the encoded format specified via the MessageEncoding field.

 

 

dummy.gifuploading.4e448015.gif转存失败重新上传取消ComponentStandardTrailer checkmark.gifuploading.4e448015.gif转存失败重新上传取消

 

Logon消息故名思议就是登入用的消息.

MsgType=A(35=A);

一个封装的例子:

 

package cs.mina.codec.msg;

import java.util.HashSet;
import java.util.List;
import java.util.Set;

import cs.mina.exception.InValidDataException;

/*
 *@author(huangxiaoping)
 *@date 2013-10-25
 */
public class LogonMsg extends BaseMsg {
	private Tag encryptMethod=new Tag("98","int",true);
	private Tag heartBtInt=new Tag("108","int",true);
	private Tag rawDataLength=new Tag("95","Length",false);
	private Tag rawData=new Tag("96","data",false);
	private Tag resetSeqNumFlag=new Tag("141","Boolean",false);
	private Tag nextExpectedMsgSeqNum=new Tag("789","SeqNum",true);
	private Tag maxMessageSize=new Tag("383","Length",false);
	private Tag msgTypeGrpTag=new MsgTypeGrpTag();
	private Tag testMessageIndicator=new Tag("464","Boolean",false);
	private Tag username=new Tag("553","String",false);
	private Tag password=new Tag("554","String",false);
	private Tag defaultApplVerID=new Tag("1137","String",true);
	private Tag text=new Tag("58","String",false);
	private Tag newPassword=new Tag("529","String",false);
	private Tag encryptedPasswordMethod=new Tag("1400","int",false);
	private Tag encryptedPasswordLen=new Tag("1401","Length",false);
	private Tag encryptedPassword=new Tag("1402","data",false);
	private Tag encryptedNewPasswordLen=new Tag("1403","Length",false);
	private Tag encryptedNewPassword=new Tag("1404","data",false);
	private Tag SessionStatus=new Tag("1409","int",false);
	private Tag defaultApplExtID=new Tag("1407","int",false);
	private Tag defaultCstmApplVerID=new Tag("1408","String",false);
	private Tag encodedTextLen=new Tag("354","Length",false);
	private Tag encodedText=new Tag("355","data",false);
	
	public LogonMsg(){
		this.getHeadEntity().getMsgType().setTagValue("A");
		tagIdsSet.add("98");
		tagIdsSet.add("108");
		tagIdsSet.add("95");
		tagIdsSet.add("96");
		tagIdsSet.add("141");
		tagIdsSet.add("789");
		tagIdsSet.add("383");
		tagIdsSet.add("464");
		tagIdsSet.add("553");
		tagIdsSet.add("554");
		tagIdsSet.add("1137");
		tagIdsSet.add("372");
		tagIdsSet.add("385");
		tagIdsSet.add("1130");
		tagIdsSet.add("1131");
		tagIdsSet.add("384");
		tagIdsSet.add("58");
		tagIdsSet.add("529");
		tagIdsSet.add("1400");
		tagIdsSet.add("1401");
		tagIdsSet.add("1402");
		tagIdsSet.add("1403");
		tagIdsSet.add("1404");
		tagIdsSet.add("1409");
		tagIdsSet.add("1407");
		tagIdsSet.add("1408");
		tagIdsSet.add("354");
		tagIdsSet.add("355");
		this.bodyEntity.getBodyTagList().add(encryptMethod);
		this.bodyEntity.getBodyTagList().add(heartBtInt);
		this.bodyEntity.getBodyTagList().add(rawDataLength);
		this.bodyEntity.getBodyTagList().add(rawData);
		this.bodyEntity.getBodyTagList().add(resetSeqNumFlag);
		this.bodyEntity.getBodyTagList().add(nextExpectedMsgSeqNum);
		this.bodyEntity.getBodyTagList().add(maxMessageSize);
		this.bodyEntity.getBodyTagList().add(msgTypeGrpTag);
		this.bodyEntity.getBodyTagList().add(testMessageIndicator);
		this.bodyEntity.getBodyTagList().add(username);
		this.bodyEntity.getBodyTagList().add(password);
		this.bodyEntity.getBodyTagList().add(defaultApplVerID);
		this.bodyEntity.getBodyTagList().add(text);
		this.bodyEntity.getBodyTagList().add(newPassword);
		this.bodyEntity.getBodyTagList().add(encryptedPasswordMethod);
		this.bodyEntity.getBodyTagList().add(encryptedPasswordLen);
		this.bodyEntity.getBodyTagList().add(encryptedPassword);
		this.bodyEntity.getBodyTagList().add(encryptedNewPasswordLen);
		this.bodyEntity.getBodyTagList().add(encryptedNewPassword);
		this.bodyEntity.getBodyTagList().add(SessionStatus);
		this.bodyEntity.getBodyTagList().add(defaultApplExtID);
		this.bodyEntity.getBodyTagList().add(defaultCstmApplVerID);
		this.bodyEntity.getBodyTagList().add(encodedTextLen);
		this.bodyEntity.getBodyTagList().add(encodedText);
	}
	
	private Set<String> tagIdsSet=new HashSet<String>();
	
	@Override
	public void validate() {
		this.headEntity.validate();
		List<Tag> bodyTagList=this.bodyEntity.getBodyTagList();
		for(int i=0;i<bodyTagList.size();i++){
			bodyTagList.get(i).validate();
		}
		this.tailerEntity.validate();
		int encryptMethodType= Integer.parseInt(encryptMethod.getTagValue());
		if(encryptMethodType<0||encryptMethodType>6){
			throw new InValidDataException("encryptMethod错误");
		}
		if(!MsgUtil.applVerID.contains(defaultApplVerID.getTagValue())){
			throw new InValidDataException("defaultApplVerID错误");
		}
		if(SessionStatus.getTagValue()!=null){
			if(!MsgUtil.sessionStatus.contains(SessionStatus.getTagValue())){
				throw new InValidDataException("sessionStatus错误");
			}
		}
		

	}

	public Tag getEncryptMethod() {
		return encryptMethod;
	}

	public void setEncryptMethod(Tag encryptMethod) {
		this.encryptMethod = encryptMethod;
	}

	public Tag getHeartBtInt() {
		return heartBtInt;
	}

	public void setHeartBtInt(Tag heartBtInt) {
		this.heartBtInt = heartBtInt;
	}

	public Tag getRawDataLength() {
		return rawDataLength;
	}

	public void setRawDataLength(Tag rawDataLength) {
		this.rawDataLength = rawDataLength;
	}

	public Tag getRawData() {
		return rawData;
	}

	public void setRawData(Tag rawData) {
		this.rawData = rawData;
	}

	public Tag getResetSeqNumFlag() {
		return resetSeqNumFlag;
	}

	public void setResetSeqNumFlag(Tag resetSeqNumFlag) {
		this.resetSeqNumFlag = resetSeqNumFlag;
	}

	public Tag getNextExpectedMsgSeqNum() {
		return nextExpectedMsgSeqNum;
	}

	public void setNextExpectedMsgSeqNum(Tag nextExpectedMsgSeqNum) {
		this.nextExpectedMsgSeqNum = nextExpectedMsgSeqNum;
	}

	public Tag getMaxMessageSize() {
		return maxMessageSize;
	}

	public void setMaxMessageSize(Tag maxMessageSize) {
		this.maxMessageSize = maxMessageSize;
	}

	public Tag getMsgTypeGrpTag() {
		return msgTypeGrpTag;
	}

	public void setMsgTypeGrpTag(Tag msgTypeGrpTag) {
		this.msgTypeGrpTag = msgTypeGrpTag;
	}

	public Tag getTestMessageIndicator() {
		return testMessageIndicator;
	}

	public void setTestMessageIndicator(Tag testMessageIndicator) {
		this.testMessageIndicator = testMessageIndicator;
	}

	public Tag getUsername() {
		return username;
	}

	public void setUsername(Tag username) {
		this.username = username;
	}

	public Tag getPassword() {
		return password;
	}

	public void setPassword(Tag password) {
		this.password = password;
	}

	public Tag getDefaultApplVerID() {
		return defaultApplVerID;
	}

	public void setDefaultApplVerID(Tag defaultApplVerID) {
		this.defaultApplVerID = defaultApplVerID;
	}

	@Override
	public void decodeBody() {
		Set<String> already=new HashSet<String>();
		String[] bodyItems=this.body.split(BaseMsg.SOH);
		for(int i=0;i<bodyItems.length;i++){
			String[]tagItems=bodyItems[i].split("=");
			if(tagItems.length!=2){
				throw new InValidDataException("消息格式错误");
			}
			String tagId=tagItems[0];
			if(already.contains(tagId)){
				throw new InValidDataException("消息格式错误");
			}
			already.add(tagId);
			if(this.tagIdsSet.contains(tagId)){
				if(tagId.equals("384")){
					int skip=decodeGroup384(this.body);
					i=i+skip-1;
				}else{
					List<Tag> tagList=this.bodyEntity.getBodyTagList();
					for(int j=0;j<tagList.size();j++){
						Tag tag=tagList.get(j);
						if(tag.getTagId().equals(tagId)){
							tag.setTagValue(tagItems[1]);
							break;
						}
					}
				}
			}else{
				throw new InValidDataException("消息格式错误,"+tagId+"不在消息体内");
			}
		}
		
	}
	
	private int decodeGroup384(String body){
		int index=body.indexOf("384=");
		String rest=body.substring(index);
		String[]tagItems=rest.split(BaseMsg.SOH);
		String[] numTagItems=tagItems[0].split("=");
		if(numTagItems.length!=2){
			throw new InValidDataException("消息格式错误");
		}
		if(!"384".equals(numTagItems[0])){
			throw new InValidDataException("消息格式错误");
		}
		int num=Integer.parseInt(numTagItems[1]);
		int count=1;
		int whileCount=0;
		((MsgTypeGrpTag)this.msgTypeGrpTag).setTagValue(num+"");
		boolean end=false;
		for(int i=0;i<num;i++){
			if(end){
				if(i!=num){
					throw new InValidDataException("消息格式错误");
				}else{
					break;
				}
			}
			MsgTypeGrp msgTypeGrp=new MsgTypeGrp();
			((MsgTypeGrpTag)this.msgTypeGrpTag).getChildren().add(msgTypeGrp);
			while(true){
				if(whileCount>=4){
					whileCount=0;
					break;
				}
				String[] tagIt=tagItems[count].split("=");
				if(tagIt.length!=2){
					throw new InValidDataException("消息格式错误");
				}
				if(whileCount==0){
					if(!MsgTypeGrpTag.first.equals(tagIt[0])){
						throw new InValidDataException("消息格式错误");
					}
				}
				
				if(MsgTypeGrp.tagIdsSet.contains(tagIt[0])){
					List<Tag> tagList=msgTypeGrp.getTagList();
					for(int j=0;j<tagList.size();j++){
						Tag groupTag=tagList.get(j);
						if(groupTag.getTagId().equals(tagIt[0])){
							groupTag.setTagValue(tagIt[1]);
							break;
						} 
					}
					if(tagItems.length>count+1){
						String[] nextTagIt=tagItems[count+1].split("=");
						if(nextTagIt.length!=2){
							throw new InValidDataException("消息格式错误");
						}
						if(MsgTypeGrpTag.first.equals(nextTagIt[0])){
							whileCount=0;
							count++;
							break;
						}
					}
					
					count++;
					whileCount++;
					
				}else{
					end=true;
					break;
				}
				
			}
			
		}
		
		return count;
	}

	public Set<String> getTagIdsSet() {
		return tagIdsSet;
	}

	public void setTagIdsSet(Set<String> tagIdsSet) {
		this.tagIdsSet = tagIdsSet;
	}

	public Tag getText() {
		return text;
	}

	public void setText(Tag text) {
		this.text = text;
	}

	public Tag getNewPassword() {
		return newPassword;
	}

	public void setNewPassword(Tag newPassword) {
		this.newPassword = newPassword;
	}

	public Tag getEncryptedPasswordMethod() {
		return encryptedPasswordMethod;
	}

	public void setEncryptedPasswordMethod(Tag encryptedPasswordMethod) {
		this.encryptedPasswordMethod = encryptedPasswordMethod;
	}

	public Tag getEncryptedPasswordLen() {
		return encryptedPasswordLen;
	}

	public void setEncryptedPasswordLen(Tag encryptedPasswordLen) {
		this.encryptedPasswordLen = encryptedPasswordLen;
	}

	public Tag getEncryptedPassword() {
		return encryptedPassword;
	}

	public void setEncryptedPassword(Tag encryptedPassword) {
		this.encryptedPassword = encryptedPassword;
	}

	public Tag getEncryptedNewPasswordLen() {
		return encryptedNewPasswordLen;
	}

	public void setEncryptedNewPasswordLen(Tag encryptedNewPasswordLen) {
		this.encryptedNewPasswordLen = encryptedNewPasswordLen;
	}

	public Tag getEncryptedNewPassword() {
		return encryptedNewPassword;
	}

	public void setEncryptedNewPassword(Tag encryptedNewPassword) {
		this.encryptedNewPassword = encryptedNewPassword;
	}

	public Tag getSessionStatus() {
		return SessionStatus;
	}

	public void setSessionStatus(Tag sessionStatus) {
		SessionStatus = sessionStatus;
	}

	public Tag getDefaultApplExtID() {
		return defaultApplExtID;
	}

	public void setDefaultApplExtID(Tag defaultApplExtID) {
		this.defaultApplExtID = defaultApplExtID;
	}

	public Tag getDefaultCstmApplVerID() {
		return defaultCstmApplVerID;
	}

	public void setDefaultCstmApplVerID(Tag defaultCstmApplVerID) {
		this.defaultCstmApplVerID = defaultCstmApplVerID;
	}

	public Tag getEncodedTextLen() {
		return encodedTextLen;
	}

	public void setEncodedTextLen(Tag encodedTextLen) {
		this.encodedTextLen = encodedTextLen;
	}

	public Tag getEncodedText() {
		return encodedText;
	}

	public void setEncodedText(Tag encodedText) {
		this.encodedText = encodedText;
	}

}


数据例子:

 

 

输入:
8=FIXT.1.1_9=86_35=A_49=CO99999901_56=HKEXCO_34=100_52=20131126-16:44:17.542_98=0_108=5_789=11_1137=9_10=028_

输出:
8=FIXT.1.1_9=117_35=A_49=HKEXCO_56=CO99999901_34=13_52=20131126-08:45:59.135_1128=9_58=Session Active_98=0_108=20_789=1_1137=9_1409=0_10=166_8=FIXT.1.1_9=110_35=4_49=HKEXCO_56=CO99999901_34=11_52=20131126-08:45:59.135_1128=9_43=Y_122=20131126-08:45:59.135_36=14_123=Y_10=024_


特别字段说明:

 

 

 

from tkinter import * from tkinter.messagebox import * import time from bs4 import BeautifulSoup from selenium import webdriver from selenium.webdriver.support.wait import WebDriverWait from selenium.webdriver.support import expected_conditions as EC from selenium.webdriver.common.by import By from selenium.common.exceptions import ElementClickInterceptedException import json import datetime # 能輸fb帳號密碼的功能 def login_fb(page_url, your_email, your_pw): options = webdriver.ChromeOptions() prefs = {'profile.default_content_setting_values': {'notifications' : 2}} options.add_experimental_option('prefs', prefs) # 版权声明:本文为优快云博主「测试小李」的原创文章,遵循 CC 4.0 BY-SA 版权协议,转载请附上原文出处链接及本声明。 # 原文链接:https://blog.youkuaiyun.com/andydufre/article/details/79204158 driver = webdriver.Chrome(options = options, executable_path="chromedriver.exe") # driver.implicitly_wait(5) driver.get(page_url) count = 0 for i in range(3): email = driver.find_element_by_id("email") email.send_keys(your_email) password = driver.find_element_by_id("pass") password.send_keys(your_pw) try: driver.find_element_by_id("u_0_2").click() except: driver.find_element_by_id("loginbutton").click() # 如果沒有成功則重新書帳號密碼, 若成功則繼續下一步 time.sleep(1) try: # 確認是否能找到email及 password標籤,若找地到代表沒有成功 if driver.find_element_by_id("email") == e_mail.get() and driver.find_element_by_id("pass") == e_mail.get(): driver.find_element_by_id("u_0_2").click() else: driver.find_element_by_id("email").clear() driver.find_element_by_id("pass").clear() count +=1 print("失敗,嘗試重新: {}次".format(count)) except: break return driver def change_lang(driver): driver.execute_script("window.scrollTo(0,1000)") langs = driver.find_elements_by_class_name("_5f4c") for lang in langs: if lang.text == "English (US)": lang.click() button = WebDriverWait(driver, 5, 0.5).until(EC.element_to_be_clickable((By.CLASS_NAME, "layerConfirm"))) button.click() # 取得所有貼文的資訊 def get_posts(driver): soup = BeautifulSoup(driver.page_source, "html5lib") posts = soup.find_all("div", {"class":"_5pcr userContentWrapper"}) return posts def get_post_timeObj(post): timeString = post.find("abbr", "_5ptz")['title'] timeObj = datetime.datetime.strptime(timeString, "%A, %B %d, %Y at %I:%M %p") return timeObj # 其功能為抓取當下driver取得的畫面中的貼文日期,將日期由字串轉為日期物件,並放到一個串列儲存並返回 def get_post_time(driver): soup = BeautifulSoup(driver.page_source, "html5lib") posts_head = soup.find_all("div", "clearfix g_5ogezyw9k") post_dateObj_list = [] for post in posts_head: post_dateObj = get_post_timeObj(post) post_dateObj_list.append(post_dateObj) return post_dateObj_list def timeStr_to_timeOBj(string): timeObj = datetime.datetime.strptime(string, "%Y-%m-%d") return timeObj # 功能為判斷driver取得的畫面中的貼文是否在所指定的時間中,若沒有則自動向下滾動取得更多貼文 def get_posts_of_period(driver, post_dateObj_list, start_date): # 將參數:起始日期、結束日期轉為日期物件 start_dateObj = timeStr_to_timeOBj(start_date) current_post_dateObj_list = post_dateObj_list # 判斷 起訖日期是否在目前畫面中, 變數為布林 while True: try: is_start_date_in_list = start_dateObj >= current_post_dateObj_list[-1] if is_start_date_in_list: break else: # execute_script 可讓網頁滾到最下面,得到更多貼文 driver.execute_script('window.scrollTo(0, document.body.scrollHeight);') current_post_dateObj_list = get_post_time(driver) time.sleep(2) except Exception as e: print(e) print("Trying to fix the error...") time.sleep(1) current_post_dateObj_list = get_post_time(driver) def filter_post_of_period(posts, start_date, end_date): start_dateObj = timeStr_to_timeOBj(start_date) end_dateObj = timeStr_to_timeOBj(end_date) posts_of_period = [] for post in posts: post_dateObj = get_post_timeObj(post) if (end_dateObj + datetime.timedelta(days=1)) > post_dateObj >= start_dateObj: # end_dateObj + datetime.timedelta(days=1) 代表 end date 的一整天 posts_of_period.append(post) return posts_of_period def expand(driver, n): for i in range(n): print("第 {} 次展開".format(i+1)) time.sleep(5) for i in driver.find_elements_by_class_name("_4ssp"): print(i.text) try: i.click() time.sleep(2) except ElementClickInterceptedException: #print(i.id, "is not clickable") driver.execute_script("arguments[0].click();", i) time.sleep(2) def get_post_infos(posts): posts_infos = [] # 貼文資訊的串列含有 日期物件、貼文內容、按讚數 for post in posts: # 用戶名 username = [name for name in post.find("h5", "_7tae").stripped_strings][0] # 將標籤中的多個字串切成串列,第一個值必為用戶ID # 貼文時間 post_time = post.find("abbr", "_5ptz")['title'] try: content = post.find("div", "_5pbx").text except: content = 'https://www.facebook.com' + post.find("div", "mtm").a['href'].split('?', 2)[0] # 按讚數 try: thumbsup = post.find("span", "_1n9k").a["aria-label"].split()[0].replace(",", "") if "K" in thumbsup: thumbsup = int(thumbsup.replace("K", "")) * 1000 else: thumbsup = int(thumbsup) except: thumbsup = 0 # 找出貼文中的留言樹及分享數 # 留言數 comments_and_shares_block = post.find("div", "_4vn1") try: amount_comments = comments_and_shares_block.find("a", "_3hg- _42ft").text.split(" ")[0] if "K" in amount_comments: amount_comments = float(amount_comments.replace("K", "")) * 1000 amount_comments = round(amount_comments) # round() 去除小數 else: amount_comments = int(amount_comments) except: amount_comments = 0 # 分享數 try: amount_shares = comments_and_shares_block.find("a", "_3rwx _42ft").text.split(" ")[0] if "K" in amount_shares: amount_shares = float(amount_shares.replace("K", "")) * 1000 amount_shares = round(amount_shares) # round() 去除小數 else: amount_shares = int(amount_shares) except: amount_shares = 0 post_info = { "username": username, "time": post_time, "content": content, "thumbsup": thumbsup, "amount_comments": amount_comments, "amount_shares": amount_shares # round() 去除小數 } posts_infos.append(post_info) return posts_infos def filter_thumbsup_number(posts_infos, number): popular_posts = [] for post in posts_infos: thumbsup_number = post["thumbsup"] try: if thumbsup_number >= number: popular_posts.append(post) except ValueError: print("Invalid post", post) print("超過 {} 個讚的貼文: {:d} 則".format(number, len(popular_posts))) return popular_posts def save_json(data): if json_name.get() == "": print("未輸JSON檔名,不會儲存資料") else: print("將抓到的資料儲存至 {} 中...".format(json_name.get())) f = open(json_name.get(), "w", encoding="utf-8") json.dump(data, f, indent=2, ensure_ascii=False) f.close() def collecting_comments(posts): posts_with_comments = [] count_post_having_comments = 0 for post in posts: post_time = post.find("abbr", "_5ptz")['title'] # 貼文中的每一條留言 comments_of_post = post.find_all("div", "_6qw3") if comments_of_post: try: post_message = post.find("div", "_5pbx").text except: post_message = 'https://www.facebook.com' + post.find("div", "mtm").a['href'].split('?', 2)[0] print("尋找 {} 中的留言...".format(post_message)) # 將貼文中的所有留言以dict格式儲存 post_comments = {} for cmt in comments_of_post: # 每條留言的留言者ID username = cmt.find("a", "_6qw4").text comments = [] for i in range(3): try: comment = cmt.find("span", "_3l3x").text comments.append(comment) break except Exception as e: print(e) print("Trying to fix the error...") if username in post_comments: post_comments[username] += comments else: post_comments[username] = comments count_post_having_comments += 1 else: continue # 將所有貼文以串列的方式儲存 posts_with_comments.append({ "post message": post_message, "post time": post_time, "comments": post_comments }) return posts_with_comments, count_post_having_comments def show_comments_data(): driver = login_fb(url.get(), e_mail.get(), password.get()) # 將 Facebook地顯示語言換成英文 change_lang(driver) # 如果是社團首頁則轉換成最近貼文先顯示 for i in range(3): try: driver.find_element_by_link_text("NEW ACTIVITY").click() driver.find_elements_by_class_name("_54nc")[1].click() break except: continue # 如果是粉專首頁則轉換到貼文區 if driver.find_elements_by_class_name("_2yau"): for area in driver.find_elements_by_class_name("_2yau"): if area.text == "Posts": area.click() break # 取的畫面中的貼文時間字串,將字串轉換成日期物件並放一個串列 post_time_object_list = get_post_time(driver) # 找出起始日期到結束日期的貼文 get_posts_of_period(driver, post_time_object_list, start_date.get()) # 展開留言中的更多留言、回覆 expand(driver, 2) time.sleep(2) posts = get_posts(driver) posts_of_period = filter_post_of_period(posts, start_date.get(), end_date.get()) posts_with_comments, count_post_having_comments = collecting_comments(posts_of_period) print("共有 {} 則有流言的貼文".format(count_post_having_comments)) # 判斷是否有增加姓名條件 if user_name.get() != "": posts_with_someone_coments = [] for post in posts_with_comments: someone_comments = {} post_comments = post["comments"] if user_name.get() in post_comments: someone_comments[user_name.get()] = post_comments[user_name.get()] post["comments"] = someone_comments posts_with_someone_coments.append(post) posts_with_comments = posts_with_someone_coments if len(posts_with_comments) == 0: showinfo("重要訊息", "沒有任何{}的留言".format(user_name.get())) else: # 印出貼文 print("找到含有 {} 流言的貼文:{} 則".format(user_name.get(), len(posts_with_comments))) for post in posts_with_comments: print(post, end="\n") # 儲存資料到json檔 save_json(posts_with_comments) else: save_json(posts_with_comments) driver.quit() def show_post_data(): # fb driver = login_fb(url.get(), e_mail.get(), password.get()) # 將 Facebook地顯示語言換成英文 change_lang(driver) # 如果是社團首頁則轉換成最近貼文先顯示 for i in range(3): try: time.sleep(1) driver.find_element_by_link_text("NEW ACTIVITY").click() driver.find_elements_by_class_name("_54nc")[1].click() time.sleep(2) break except: continue # 如果是粉專首頁則轉換到貼文區 if driver.find_elements_by_class_name("_2yau"): for area in driver.find_elements_by_class_name("_2yau"): if area.text == "Posts": area.click() break # 取的畫面中的貼文時間字串,將字串轉換成日期物件並放一個串列 post_time_object_list = get_post_time(driver) # 找出起始日期到結束日期的貼文 time.sleep(2) get_posts_of_period(driver, post_time_object_list, start_date.get()) posts = get_posts(driver) posts_of_period = filter_post_of_period(posts, start_date.get(), end_date.get()) posts_infos = get_post_infos(posts_of_period) # 判斷是否有輸按讚數條件 if thumbsup_number.get() > 0: posts_infos = filter_thumbsup_number(posts_infos, thumbsup_number.get()) # 判斷是否有輸姓名條件 if user_name.get() != "": posts_infos = [post for post in posts_infos if post["username"] == user_name.get()] if len(posts_infos) == 0: showinfo("重要訊息", "沒有符合條件的貼文") else: # 印出貼文 print("找到符合條件的貼文:{} 則".format(len(posts_infos))) for post in posts_infos: print(post, end="\n") # 儲存資料到json檔 save_json(posts_infos) driver.quit() if __name__ == '__main__': # 建立Tk物件, 設定視窗大小 window = Tk() window.title("Facebook爬蟲程式") window.geometry("600x200") window.maxsize(900,500) # 輸值變數 e_mail = StringVar() password = StringVar() url = StringVar() user_name = StringVar() start_date = StringVar() end_date = StringVar() thumbsup_number = IntVar() json_name = StringVar() # 標籤元件設定 email_label = Label(window, text="帳號:").grid(row=0, column=0, sticky=E) password_label = Label(window, text="密碼:").grid(row=0, column=2, sticky=E) url_label = Label(window, text="網址:").grid(row=1, sticky=E) setting_label = Label(window, text="條件設定:").grid(row=3,sticky=W) username_label = Label(window, text="姓名:").grid(row=4, sticky=E) start_date_label= Label(window, text="日期:從").grid(row=5, sticky=E) end_date_label= Label(window, text="至").grid(row=5, column=2, sticky=W+E+S+N) thumbsup_number_label = Label(window, text="按讚數:").grid(row=6, sticky=W) save_json_name_label = Label(window, text="檔名:").grid(row=7, column=0, sticky=W) # 輸欄元件設定 input_email = Entry(window, textvariable=e_mail).grid(row=0, column=1, sticky=W) input_password = Entry(window, textvariable=password, show="*").grid(row=0, column=3, sticky=W) input_url = Entry(window, width=70, textvariable=url).grid(row=1, column=1, columnspan=50, sticky=W) input_username = Entry(window, textvariable=user_name).grid(row=4, column=1, sticky=W) input_startdate = Entry(window, textvariable=start_date).grid(row=5, column=1, sticky=W) input_enddate = Entry(window, textvariable=end_date).grid(row=5, column=3, sticky=W) input_thumbsup_number = Entry(window, textvariable=thumbsup_number).grid(row=6, column=1, sticky=W) input_json_name = Entry(window, textvariable=json_name).grid(row=7, column=1, sticky=W) #按鈕元件設定,觸發程式執行 button1 = Button(window, text="尋找貼文", command=show_post_data).grid(row=8, column=1) button2 = Button(window, text="尋找留言", command=show_comments_data).grid(row=8, column=3) window.mainloop()
最新发布
12-21
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

hxpjava1

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值