The Google File System : part7 EXPERIENCES

7. EXPERIENCES
In the process of building and deploying GFS, we have experienced a variety of issues, some operational and some technical.

Initially, GFS was conceived as the backend file system for our production systems. 
Over time, the usage evolved to include research and development tasks. 
It started with little support for things like permissions and quotas but now includes rudimentary forms of these. 
While production systems are well disciplined and controlled, users sometimes are not. 
More infrastructure is required to keep users from interfering with one another.
Some of our biggest problems were disk and Linux related.
Many of our disks claimed to the Linux driver that they supported a range of IDE protocol versions but in fact responded reliably only to the more recent ones. 
Since the protocol versions are very similar, these drives mostly worked,but occasionally the mismatches would cause the drive and the kernel to disagree about the drive’s state. 
This would corrupt data silently due to problems in the kernel. 
This problem motivated our use of checksums to detect data corruption, while concurrently we modified the kernel to handle these protocol mismatches.
Earlier we had some problems with Linux 2.2 kernels due to the cost of fsync(). 
Its cost is proportional to the size of the file rather than the size of the modified portion. 
This was a problem for our large operation logs especially before we implemented checkpointing. 
We worked around this for a time by using synchronous writes and eventually migrated to Linux 2.4.

Another Linux problem was a single reader-writer lock which any thread in an address space must hold when it pages in from disk (reader lock) or modifies the address space in an mmap() call (writer lock). 
We saw transient timeouts in our system under light load and looked hard for resource bottlenecks or sporadic hardware failures. 
Eventually, we found that this single lock blocked the primary network thread from mapping new data into memory while the disk threads were paging in previously mapped data.
Since we are mainly limited by the network interface rather than by memory copy bandwidth, we worked around this by replacing mmap() with pread() at the cost of an extra copy.
Despite occasional problems, the availability of Linux code has helped us time and again to explore and understand system behavior. 
When appropriate, we improve the kernel and share the changes with the open source community.

7.经验
在建设和部署GFS的过程中,我们经历了各种问题,一些操作和一些技术。
最初,GFS被认为是我们生产系统的后端文件系统。
随着时间的推移,使用演变成包括研发任务。
它开始于对许可和配额等方面的支持,但现在包括这些的基本形式。
虽然生产系统规范和控制严格,但用户有时并不是这样。
需要更多的基础设施来防止用户互相干扰。
我们最大的一些问题是磁盘和Linux相关。
我们的许多磁盘声称对Linux驱动程序,他们支持一系列IDE协议版本,但实际上只是对最近的响应可靠。
由于协议版本非常相似,这些驱动器主要工作,但有时不匹配将导致驱动器和内核对驱动器的状态不同意。
这将由于内核中的问题而静默地破坏数据。
这个问题促使我们使用校验和来检测数据损坏,同时我们修改了内核来处理这些协议不匹配。
早些时候,由于fsync()的成本,Linux 2.2内核出现了一些问题。
其成本与文件的大小成正比,而不是修改部分的大小。
这对我们的大型操作日志是一个问题,特别是在我们实现检查点之前。
我们通过使用同步写入操作了一段时间,最终迁移到Linux 2.4。
另一个Linux问题是一个单一的读写器锁,地址空间中的任何线程在从磁盘(读卡器锁定)中进行访问或修改mmap()调用(写入器锁)中的地址空间时都必须保持。
我们在轻载下看到我们的系统中出现瞬时超时,并且很难看出资源瓶颈或零星的硬件故障。
最终,我们发现这个单一的锁阻止主网络线程将新数据映射到内存,而磁盘线程在以前映射的数据中进行分页。
由于我们主要受网络接口的限制,而不是内存复制带宽限制,我们通过以额外的副本为代价替换了mmap()与pread()。
尽管偶尔出现问题,但是Linux代码的可用性帮助我们一次又一次地探索和理解系统行为。
在适当的情况下,我们改进内核并与开源社区共享这些更改。
### Selenium 4.27 Compatible Google ChromeDriver Download and Usage For ensuring compatibility between Selenium 4.27 and the corresponding version of ChromeDriver, it is crucial to match versions carefully[^1]. The error message `selenium.common.exceptions.WebDriverException: Message: 'chromedriver' executable needs to be in PATH` indicates that Python cannot locate the ChromeDriver executable file on your system. To resolve this issue: #### Verify Installed Chrome Browser Version Firstly, determine which version of Chrome browser you have installed since ChromeDriver must correspond with the major version number of the Chrome installation. For instance, a Chrome browser at version 104.x requires a matching ChromeDriver release such as 104.0.5112.79[^3]. #### Obtain Correct ChromeDriver Version Navigate to the official or mirrored repository where ChromeDriver binaries are hosted. A reliable source can be found here: [https://registry.npmmirror.com/binary.html?path=chromedriver/](https://registry.npmmirror.com/binary.html?path=chromedriver/) . Choose the appropriate driver based upon both operating system architecture (Windows, macOS, Linux) and bitness (32-bit vs 64-bit). After downloading, extract the contents from any archive files provided by the website into an accessible directory within your project folder structure or another preferred location on disk storage devices connected to your computer setup environment configuration settings path variable directories list entries array elements collection set items group members ensemble components part constituents pieces fragments sections segments divisions parts portions fractions proportions shares slices chunks lumps masses bodies entities objects items units specimens samples instances cases examples illustrations representations manifestations expressions embodiments incarnations avatars personifications exemplars paradigms models patterns prototypes archetypes templates blueprints schematics diagrams charts graphs maps layouts designs structures frameworks architectures systems networks webs meshes grids matrices arrays tables lists sequences strings chains links rings loops circles cycles circuits paths routes ways roads streets lanes avenues boulevards highways freeways expressways motorways parkways causeways bridges tunnels crossings intersections junctions nodes points spots locations places sites positions coordinates addresses destinations targets goals endpoints termini stations stops platforms quays piers docks harbors ports airports terminals depots garages yards lots fields grounds spaces areas zones regions territories domains realms worlds universes cosmos galaxies constellations clusters groups bunches heaps piles stacks columns rows lines ranks orders hierarchies levels layers stages steps degrees grades classes categories types kinds sorts varieties species genera families orders classes phyla kingdoms domains life forms beings creatures organisms entities existences realities phenomena events occurrences happenings incidents accidents experiences encounters meetings confrontations interactions exchanges transactions transfers transmissions deliveries distributions circulations movements motions actions operations procedures processes methods techniques approaches strategies tactics plans schemes projects initiatives efforts attempts trials experiments studies research investigations explorations inquiries queries questions interrogations examinations inspections observations perceptions recognitions identifications diagnoses analyses interpretations explanations descriptions narratives stories tales accounts reports records documents writings texts scripts codes programs software applications tools utilities aids assistants helpers supporters promoters advocates sponsors patrons benefactors donors contributors providers suppliers vendors sellers merchants traders dealers brokers agents representatives delegates ambassadors envoys messengers couriers carriers transporters conveyors transmitters broadcasters publishers distributors disseminators spreaders sharers givers donators benefactors philanthropists humanitarians altruists do-gooders well-wishers friends allies partners colleagues teammates collaborators cooperators participants actors players performers artists creators producers directors managers administrators supervisors leaders guides mentors coaches trainers instructors teachers educators professors lecturers speakers presenters announcers narrators commentators analysts critics reviewers judges referees umpires arbiters mediators moderators facilitators coordinators organizers planners designers architects engineers builders constructors manufacturers producers fabricators makers creators inventors innovators pioneers trailblazers groundbreakers trendsetters leaders visionaries dreamers idealists optimists realists pragmatists materialists empiricists rationalists logicians thinkers philosophers scholars academics intellectuals professionals experts specialists authorities figures icons legends heroes champions victors winners success stories role models inspirations motivations forces powers influences impacts effects changes transformations revolutions evolutions developments advancements progressions expansions extensions enlargements augmentations enhancements improvements benefits advantages gains profits earnings rewards returns results outcomes consequences effects impacts impressions marks stamps signatures seals brands labels tags markers indicators signs symbols emblems badges logos crests coats-of-arms shields banners flags pennants streamers ribbons tapes threads strands fibers filaments wires cables ropes cords strings lines bands strips tapes films sheets plates panels boards cards tickets vouchers coupons receipts invoices bills statements ledgers journals logs diaries notebooks albums scrapbooks portfolios binders folders envelopes packets packages containers boxes crates barrels drums kegs casks vats tanks reservoirs pools ponds lakes rivers streams creeks brooks rivulets rills trickles drips drops splashes sprays mists fogs clouds vapors gases liquids solids plasmas energies matters substances materials resources assets properties possessions holdings estates dominions jurisdictions sovereignties governments regimes administrations bureaucracies organizations institutions establishments enterprises businesses companies corporations firms agencies bureaus offices departments ministries councils committees commissions boards tribunals courts parliaments congresses legislatures assemblies conventions synods convocations gatherings meetings conferences summ
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值