DataStage:DataStage Parallel Job VS Server Job

本人英文水平一般,先收着以后再翻译吧。
1) The basic difference between server and parallel jobs is the degree of parallelism
Server job stages do not have in built partitoning and parallelism mechanism for extracting and loading data between different stages. 

All you can do to enhance the speed and perormance in server jobs is to enable inter process row buffering through the administrator. This helps stages to exchange data as soon as it is available in the link. 
You could use IPC stage too which helps one passive stage read data from another as soon as data is available. In other words, stages do not have to wait for the entire set of records to be read first and then transferred to the next stage. Link partitioner and link collector stages can be used to achieve a certain degree of partitioning paralellism. 
All of the above features which have to be explored in server jobs are built in datastage Px. 

2) The Px engine runs on a multiprocessor system and takes full advantage of the processing nodes defined in the configuration file. Both SMP and MMP architecture is supported by datastage Px. 

3) Px takes advantage of both pipeline parallelism and partitoning paralellism. Pipeline parallelism means that as soon as data is available between stages( in pipes or links), it can be exchanged between them without waiting for the entire record set to be read. Partitioning parallelism means that entire record set is partitioned into small sets and processed on different nodes(logical processors). For example if there are 100 records, then if there are 4 logical nodes then each node would process 25 records each. This enhances the speed at which loading takes place to an amazing degree. Imagine situations where billions of records have to be loaded daily. This is where datastage PX comes as a boon for ETL process and surpasses all other ETL tools in the market.

4) In parallel we have Dataset which acts as the intermediate data storage in the linked list, it is the best storage option it stores the data in datastage internal format.

5) In parallel we can choose to display OSH , which gives information about the how job works.

6) In Parallel Transformer there is no reference link possibility, in server stage reference could be given to transformer. Parallel stage can use both basic and parallel oriented functions.

7) Datastage server executed by datastage server environment but parallel executed under control of datastage runtime environment

8) Datastage compiled in to BASIC(interpreted pseudo code) and Parallel compiled to OSH(Orchestrate Scripting Language).

9) Debugging and Testing Stages are available only in the Parallel Extender.

10) More Processing stages are not included in Server example, Join, CDC, Lookup etc…..

11) In File stages, Hash file available only in Server and Complex falat file , dataset , lookup file set avail in parallel only.

12) Server Transformer supports basic transforms only, but in parallel both basic and parallel transforms.

13) Server transformer is basic language compatability, pararllel transformer is c++ language compatabillity

14) Look up of sequntial file is possible in parallel jobs 

15) . In parallel we can specify more file paths to fetch data from using 
file pattern similar to Folder stage in Server, while in server we can 
specify one file name in one O/P link. 

16). We can simulteneously give input as well as output link to a seq. file 
stage in Server. But an output link in parallel means a reject link, that 
is a link that collects records that fail to load into the sequential file 
for some reasons.

17). The difference is file size Restriction. 
Sequential file size in server is : 2GB 
Sequential file size in parallel is : No Limitation..

18). Parallel sequential file has filter options too. Where you can specify the file pattern.

转载自:http://www.360doc.com/content/09/0223/10/7362_2620736.shtml
内容概要:该研究通过在黑龙江省某示范村进行24小时实地测试,比较了燃煤炉具与自动/手动进料生物质炉具的污染物排放特征。结果显示,生物质炉具相比燃煤炉具显著降低了PM2.5、CO和SO2的排放(自动进料分别降低41.2%、54.3%、40.0%;手动进料降低35.3%、22.1%、20.0%),但NOx排放未降低甚至有所增加。研究还发现,经济性和便利性是影响生物质炉具推广的重要因素。该研究不仅提供了实际排放数据支持,还通过Python代码详细复现了排放特征比较、减排效果计算和结果可视化,进一步探讨了燃料性质、动态排放特征、碳平衡计算以及政策建议。 适合人群:从事环境科学研究的学者、政府环保部门工作人员、能源政策制定者、关注农村能源转型的社会人士。 使用场景及目标:①评估生物质炉具在农村地区的推广潜力;②为政策制定者提供科学依据,优化补贴政策;③帮助研究人员深入了解生物质炉具的排放特征和技术改进方向;④为企业研发更高效的生物质炉具提供参考。 其他说明:该研究通过大量数据分析和模拟,揭示了生物质炉具在实际应用中的优点和挑战,特别是NOx排放增加的问题。研究还提出了多项具体的技术改进方向和政策建议,如优化进料方式、提高热效率、建设本地颗粒厂等,为生物质炉具的广泛推广提供了可行路径。此外,研究还开发了一个智能政策建议生成系统,可以根据不同地区的特征定制化生成政策建议,为农村能源转型提供了有力支持。
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值