1.1. History of the UNIX System

本文回顾了UNIX系统的起源和发展历程,从最初的Bell Laboratories到加州大学伯克利分校的贡献,再到AT&T及开源社区的推动。文章详细介绍了不同版本UNIX系统的特性和演变,包括虚拟内存、网络协议的支持等关键进展。
部署运行你感兴趣的模型镜像
1.1. History of the UNIX System The UNIX system has been in wide use for over 30 years and has helped to define many areas of computing. Although numerous individuals and organizations have contributed (and still contribute) to the development of the UNIX system, this book primarily concentrates on the BSD thread of development. Bell Laboratories, which invented UNIX The Computer Systems Research Group (CSRG) at the University of California at Berkeley, which gave UNIX virtual memory and the reference implementation of TCP/IP The FreeBSD project, the NetBSD project, and the OpenBSD project, which continue the work started by the CSRG The Darwin operating system at the core of Apple's OS X. Darwin is based on FreeBSD. Origins The first version of the UNIX system was developed at Bell Laboratories in 1969 by Ken Thompson as a private research project to use an otherwise idle PDP-7. Thompson was joined shortly thereafter by Dennis Ritchie, who not only contributed to the design and implementation of the system, but also invented the C programming language. The system was completely rewritten into C, leaving almost no assembly language. The original elegant design of the system [Ritchie, 1978] and developments of the first 15 years [Ritchie, 1984a; Compton, 1985] have made the UNIX system an important and powerful operating system [Ritchie, 1987]. Ritchie, Thompson, and other early UNIX developers at Bell Laboratories had worked previously on the Multics project [Peirce, 1985; Organick, 1975], which had a strong influence on the newer operating system. Even the name UNIX is merely a pun on Multics; in areas where Multics attempted to do many tasks, UNIX tried to do only one task but do it well. The basic organization of the UNIX filesystem, the idea of using a user process for the command interpreter, the general organization of the filesystem interface, and many other system characteristics come directly from Multics. Ideas from various other operating systems, such as the Massachusetts Institute of Technology's (MIT's) CTSS, also have been incorporated. The fork operation to create new processes comes from Berkeley's GENIE (SDS-940, later XDS-940) operating system. Allowing a user to create processes inexpensively led to using one process per command rather than commands being run as procedure calls, as is done in Multics. Research UNIX The first major editions of UNIX were the Research systems from Bell Laboratories. In addition to the earliest versions of the system, these systems include the UNIX Time-Sharing System, Sixth Edition, commonly known as V6, which in 1976 was the first version widely available outside of Bell Laboratories. Systems are identified by the edition numbers of the UNIX Programmer's Manual that were current when the distributions were made. The UNIX system was distinguished from other operating systems in three important ways. It was written in a high-level language. It was distributed in source form. It provided powerful primitives normally found in only those operating systems that ran on much more expensive hardware. Most of the system source code was written in C rather than in assembly language. The prevailing belief at the time was that an operating system had to be written in assembly language to provide reasonable efficiency and to get access to the hardware. The C language itself was at a sufficiently high level to allow it to be compiled easily for a wide range of computer hardware, without its being so complex or restrictive that systems programmers had to revert to assembly language to get reasonable efficiency or functionality. Access to the hardware was provided through assembly-language stubs for the 3 percent of the operating-system functions—such as context switching—that needed them. Although the success of UNIX does not stem solely from its being written in a high-level language, the use of C was a critical first step [Kernighan & Ritchie, 1978; Kernighan & Ritchie, 1989; Ritchie et al., 1978]. Ritchie's C language is descended [Rosler, 1984] from Thompson's B language, which was itself descended from BCPL [Richards & Whitby-Strevens, 1980]. C continues to evolve [Tuthill, 1985; ISO, 1999]. The second important distinction of UNIX was its early release from Bell Laboratories to other research environments in source form. By providing source, the system's founders ensured that other organizations would be able not only to use the system, but also to tinker with its inner workings. The ease with which new ideas could be adopted into the system always has been key to the changes that have been made to it. Whenever a new system that tried to upstage UNIX came along, somebody would dissect the newcomer and clone its central ideas into UNIX. The unique ability to use a small, comprehensible system, written in a high-level language, in an environment swimming in new ideas led to a UNIX system that evolved far beyond its humble beginnings. Though recipients of the source code had to be licensed, campuswide licenses were cheaply available to universities. Thus, many people became versed in the way that UNIX worked, setting the stage for the open-source world that would follow. The third important distinction of UNIX was that it provided individual users with the ability to run multiple processes concurrently and to connect these processes into pipelines of commands. At the time, only operating systems running on large and expensive machines had the ability to run multiple processes, and the number of concurrent processes usually was controlled tightly by a system administrator. Most early UNIX systems ran on the PDP-11, which was inexpensive and powerful for its time. Nonetheless, there was at least one early port of Sixth Edition UNIX to a machine with a different architecture: the Interdata 7/32 [Miller, 1978]. The PDP-11 also had an inconveniently small address space. The introduction of machines with 32-bit address spaces, especially the VAX-11/780, provided an opportunity for UNIX to expand its services to include virtual memory and networking. Earlier experiments by the Research group in providing UNIX-like facilities on different hardware had led to the conclusion that it was as easy to move the entire operating system as it was to duplicate UNIX's services under another operating system. The first UNIX system with portability as a specific goal was UNIX Time-Sharing System, Seventh Edition (V7), which ran on the PDP-11 and the Interdata 8/32 and had a VAX variety called UNIX/32V Time-Sharing, System Version 1.0 (32V). The Research group at Bell Laboratories has also developed UNIX Time-Sharing System, Eighth Edition (V8); UNIX Time-Sharing System, Ninth Edition (V9); and UNIX Time-Sharing System, Tenth Edition (V10). Their 1996 system is Plan 9. AT&T UNIX System III and System V After the distribution of Seventh Edition in 1978, the Research group turned over external distributions to the UNIX Support Group (USG). USG had previously distributed internally such systems as the UNIX Programmer's Work Bench (PWB), and had sometimes distributed them externally as well [Mohr, 1985]. USG's first external distribution after Seventh Edition was UNIX System III (System III) in 1982, which incorporated features of Seventh Edition, of 32V, and also of several UNIX systems developed by groups other than the Research group. Features of UNIX/RT (a real-time UNIX system) were included, as were many features from PWB. USG released UNIX System V (System V) in 1983; that system is largely derived from System III. The court-ordered divestiture of the Bell Operating Companies from AT&T permitted AT&T to market System V aggressively [Bach, 1986; Wilson, 1985]. USG metamorphosed into the UNIX System Development Laboratory (USDL), which released UNIX System V, Release 2 in 1984. System V, Release 2, Version 4 introduced paging [Jung, 1985; Miller, 1984], including copy-on-write and shared memory, to System V. The System V implementation was not based on the Berkeley paging system. USDL was succeeded by AT&T Information Systems (ATTIS), which distributed UNIX System V, Release 3, in 1987. That system included STREAMS, an IPC mechanism adopted from V8 [Presotto & Ritchie, 1985]. ATTIS was succeeded by UNIX System Laboratory (USL), which was sold to Novell in 1993. Novell passed the UNIX trademark to the X/OPEN consortium, giving the latter sole rights to set up certification standards for using the UNIX name on products. Two years later, Novell sold UNIX to The Santa Cruz Operation (SCO). Berkeley Software Distributions The most influential of the non-Bell Laboratories and non-AT&T UNIX development groups was the University of California at Berkeley [DiBona et al., 1999]. Software from Berkeley was released in Berkeley Software Distributions (BSD)—for example, as 4.4BSD. Berkeley was the source of the BSD name, and their distributions were the first distinct identity for the BSD operating system. The first Berkeley VAX UNIX work was the addition to 32V of virtual memory, demand paging, and page replacement in 1979 by William Joy and Ozalp Babaoglu, to produce 3BSD [Babaoglu & Joy, 1981]. The reason for the large virtual-memory space of 3BSD was the development of what at the time were large programs, such as Berkeley's Franz LISP. This memory-management work convinced the Defense Advanced Research Projects Agency (DARPA) to fund the Berkeley team for the later development of a standard system (4BSD) for DARPA's contractors to use. A goal of the 4BSD project was to provide support for the DARPA Internet networking protocols, TCP/IP [Comer, 2000]. The networking implementation was general enough to communicate among diverse network facilities, ranging from local networks, such as Ethernets and token rings, to long-haul networks, such as DARPA's ARPANET. We refer to all the Berkeley VAX UNIX systems following 3BSD as 4BSD, although there were really several releases: 4.0BSD, 4.1BSD, 4.2BSD, 4.3BSD, 4.3BSD Tahoe, and 4.3BSD Reno. 4BSD was the UNIX operating system of choice for VAXes from the time that the VAX first became available in 1977 until the release of System V in 1983. Most organizations would purchase a 32V license but would order 4BSD from Berkeley. Many installations inside the Bell System ran 4.1BSD (and replaced it with 4.3BSD when the latter became available). A new virtual-memory system was released with 4.4BSD. The VAX was reaching the end of its useful lifetime, so 4.4BSD was not ported to that machine. Instead, 4.4BSD ran on the newer 68000, SPARC, MIPS, and Intel PC architectures. The 4BSD work for DARPA was guided by a steering committee that included many notable people from both commercial and academic institutions. The culmination of the original Berkeley DARPA UNIX project was the release of 4.2BSD in 1983; further research at Berkeley produced 4.3BSD in mid-1986. The next releases included the 4.3BSD Tahoe release of June 1988 and the 4.3BSD Reno release of June 1990. These releases were primarily ports to the Computer Consoles Incorporated hardware platform. Interleaved with these releases were two unencumbered networking releases: the 4.3BSD Netl release of March 1989 and the 4.3BSD Net2 release of June 1991. These releases extracted nonproprietary code from 4.3BSD; they could be redistributed freely in source and binary form to companies that and individuals who were not covered by a UNIX source license. The final CSRG release requiring an AT&T source license was 4.4BSD in June 1993. Following a year of litigation (see Section 1.3), the free-redistributable 4.4BSD-Lite was released in April 1994. The final CSRG release was 4.4BSD-Lite Release 2 in June 1995. UNIX in the World The UNIX system is also a fertile field for academic endeavor. Thompson and Ritchie were given the Association for Computing Machinery Turing award for the design of the system [Ritchie, 1984b]. The UNIX system and related, specially designed teaching systems—such as Tunis [Ewens et al., 1985; Holt, 1983], XINU [Comer, 1984], and MINIX [Tanenbaum, 1987]—are widely used in courses on operating systems. Linus Torvalds reimplemented the UNIX interface in his freely redistributable Linux operating system. The UNIX system is ubiquitous in universities and research facilities throughout the world, and is ever more widely used in industry and commerce.[@more@]

来自 “ ITPUB博客 ” ,链接:http://blog.itpub.net/20102458/viewspace-902808/,如需转载,请注明出处,否则将追究法律责任。

转载于:http://blog.itpub.net/20102458/viewspace-902808/

您可能感兴趣的与本文相关的镜像

Qwen3-8B

Qwen3-8B

文本生成
Qwen3

Qwen3 是 Qwen 系列中的最新一代大型语言模型,提供了一整套密集型和专家混合(MoE)模型。基于广泛的训练,Qwen3 在推理、指令执行、代理能力和多语言支持方面取得了突破性进展

2025-11-17 11:39:42,286 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: registered UNIX signal handlers for [TERM, HUP, INT] 2025-11-17 11:39:44,201 INFO org.apache.hadoop.hdfs.server.datanode.checker.ThrottledAsyncChecker: Scheduling a check for [DISK]file:/home/zwk/hadoop/hadoop-3.3.5/tmp/dfs/data 2025-11-17 11:39:44,747 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: Loaded properties from hadoop-metrics2.properties 2025-11-17 11:39:45,024 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s). 2025-11-17 11:39:45,025 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: DataNode metrics system started 2025-11-17 11:39:46,402 INFO org.apache.hadoop.hdfs.server.common.Util: dfs.datanode.fileio.profiling.sampling.percentage set to 0. Disabling file IO profiling 2025-11-17 11:39:46,463 INFO org.apache.hadoop.hdfs.server.datanode.BlockScanner: Initialized block scanner with targetBytesPerSec 1048576 2025-11-17 11:39:46,488 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Configured hostname is zwk-virtual-machine 2025-11-17 11:39:46,489 INFO org.apache.hadoop.hdfs.server.common.Util: dfs.datanode.fileio.profiling.sampling.percentage set to 0. Disabling file IO profiling 2025-11-17 11:39:46,500 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Starting DataNode with maxLockedMemory = 0 2025-11-17 11:39:46,621 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Opened streaming server at /0.0.0.0:9866 2025-11-17 11:39:46,626 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Balancing bandwidth is 104857600 bytes/s 2025-11-17 11:39:46,626 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Number threads for balancing is 100 2025-11-17 11:39:47,035 INFO org.eclipse.jetty.util.log: Logging initialized @6372ms to org.eclipse.jetty.util.log.Slf4jLog 2025-11-17 11:39:48,027 WARN org.apache.hadoop.security.authentication.server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets. Reason: Could not read signature secret file: /home/zwk/hadoop-http-auth-signature-secret 2025-11-17 11:39:48,050 INFO org.apache.hadoop.http.HttpRequestLog: Http request log for http.requests.datanode is not defined 2025-11-17 11:39:48,063 INFO org.apache.hadoop.http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2025-11-17 11:39:48,066 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context datanode 2025-11-17 11:39:48,066 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2025-11-17 11:39:48,067 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2025-11-17 11:39:48,235 INFO org.apache.hadoop.http.HttpServer2: Jetty bound to port 44301 2025-11-17 11:39:48,244 INFO org.eclipse.jetty.server.Server: jetty-9.4.48.v20220622; built: 2022-06-21T20:42:25.880Z; git: 6b67c5719d1f4371b33655ff2d047d24e171e49a; jvm 1.8.0_371-b11 2025-11-17 11:39:48,344 INFO org.eclipse.jetty.server.session: DefaultSessionIdManager workerName=node0 2025-11-17 11:39:48,344 INFO org.eclipse.jetty.server.session: No SessionScavenger set, using defaults 2025-11-17 11:39:48,353 INFO org.eclipse.jetty.server.session: node0 Scavenging every 660000ms 2025-11-17 11:39:48,404 INFO org.eclipse.jetty.server.handler.ContextHandler: Started o.e.j.s.ServletContextHandler@71529963{logs,/logs,file:///home/zwk/hadoop/hadoop-3.3.5/logs/,AVAILABLE} 2025-11-17 11:39:48,409 INFO org.eclipse.jetty.server.handler.ContextHandler: Started o.e.j.s.ServletContextHandler@3f270e0a{static,/static,file:///home/zwk/hadoop/hadoop-3.3.5/share/hadoop/hdfs/webapps/static/,AVAILABLE} 2025-11-17 11:39:48,765 INFO org.eclipse.jetty.server.handler.ContextHandler: Started o.e.j.w.WebAppContext@aafcffa{datanode,/,file:///home/zwk/hadoop/hadoop-3.3.5/share/hadoop/hdfs/webapps/datanode/,AVAILABLE}{file:/home/zwk/hadoop/hadoop-3.3.5/share/hadoop/hdfs/webapps/datanode} 2025-11-17 11:39:48,807 INFO org.eclipse.jetty.server.AbstractConnector: Started ServerConnector@74c79fa2{HTTP/1.1, (http/1.1)}{localhost:44301} 2025-11-17 11:39:48,809 INFO org.eclipse.jetty.server.Server: Started @8144ms 2025-11-17 11:39:49,100 WARN org.apache.hadoop.hdfs.server.datanode.web.DatanodeHttpServer: Got null for restCsrfPreventionFilter - will not do any filtering. 2025-11-17 11:39:49,366 INFO org.apache.hadoop.hdfs.server.datanode.web.DatanodeHttpServer: Listening HTTP traffic on /0.0.0.0:9864 2025-11-17 11:39:49,417 INFO org.apache.hadoop.util.JvmPauseMonitor: Starting JVM pause monitor 2025-11-17 11:39:49,434 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: dnUserName = zwk 2025-11-17 11:39:49,434 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: supergroup = supergroup 2025-11-17 11:39:49,782 INFO org.apache.hadoop.ipc.CallQueueManager: Using callQueue: class java.util.concurrent.LinkedBlockingQueue, queueCapacity: 1000, scheduler: class org.apache.hadoop.ipc.DefaultRpcScheduler, ipcBackoff: false. 2025-11-17 11:39:50,113 INFO org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 9867 2025-11-17 11:39:50,994 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Opened IPC server at /0.0.0.0:9867 2025-11-17 11:39:51,179 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Refresh request received for nameservices: null 2025-11-17 11:39:51,234 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Starting BPOfferServices for nameservices: <default> 2025-11-17 11:39:51,319 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Block pool <registering> (Datanode Uuid unassigned) service to localhost/127.0.0.1:9000 starting to offer service 2025-11-17 11:39:51,392 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting 2025-11-17 11:39:51,422 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 9867: starting 2025-11-17 11:39:53,925 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Acknowledging ACTIVE Namenode during handshakeBlock pool <registering> (Datanode Uuid unassigned) service to localhost/127.0.0.1:9000 2025-11-17 11:39:53,966 INFO org.apache.hadoop.hdfs.server.common.Storage: Using 1 threads to upgrade data directories (dfs.datanode.parallel.volumes.load.threads.num=1, dataDirs=1) 2025-11-17 11:39:54,005 INFO org.apache.hadoop.hdfs.server.common.Storage: Lock on /home/zwk/hadoop/hadoop-3.3.5/tmp/dfs/data/in_use.lock acquired by nodename 3605@zwk-virtual-machine 2025-11-17 11:39:54,137 INFO org.apache.hadoop.hdfs.server.common.Storage: Analyzing storage directories for bpid BP-655327536-127.0.1.1-1762954302619 2025-11-17 11:39:54,138 INFO org.apache.hadoop.hdfs.server.common.Storage: Locking is disabled for /home/zwk/hadoop/hadoop-3.3.5/tmp/dfs/data/current/BP-655327536-127.0.1.1-1762954302619 2025-11-17 11:39:54,146 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Setting up storage: nsid=1281678374;bpid=BP-655327536-127.0.1.1-1762954302619;lv=-57;nsInfo=lv=-66;cid=CID-de621368-0d95-41d0-9ebe-6538d4b7b6e3;nsid=1281678374;c=1762954302619;bpid=BP-655327536-127.0.1.1-1762954302619;dnuuid=c229f5e4-2241-43fd-8c64-6086b77fa49a 2025-11-17 11:39:54,235 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: The datanode lock is a read write lock 2025-11-17 11:39:55,179 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Added new volume: DS-5c441fde-88f8-4f39-a77a-075dfdc1cdf9 2025-11-17 11:39:55,179 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Added volume - [DISK]file:/home/zwk/hadoop/hadoop-3.3.5/tmp/dfs/data, StorageType: DISK 2025-11-17 11:39:55,198 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.MemoryMappableBlockLoader: Initializing cache loader: MemoryMappableBlockLoader. 2025-11-17 11:39:55,206 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Registered FSDatasetState MBean 2025-11-17 11:39:55,238 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Adding block pool BP-655327536-127.0.1.1-1762954302619 2025-11-17 11:39:55,245 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Scanning block pool BP-655327536-127.0.1.1-1762954302619 on volume /home/zwk/hadoop/hadoop-3.3.5/tmp/dfs/data... 2025-11-17 11:39:55,280 WARN org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: dfsUsed file missing in /home/zwk/hadoop/hadoop-3.3.5/tmp/dfs/data/current/BP-655327536-127.0.1.1-1762954302619/current, will proceed with Du for space computation calculation, 2025-11-17 11:39:55,334 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Time taken to scan block pool BP-655327536-127.0.1.1-1762954302619 on /home/zwk/hadoop/hadoop-3.3.5/tmp/dfs/data: 88ms 2025-11-17 11:39:55,335 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Total time to scan all replicas for block pool BP-655327536-127.0.1.1-1762954302619: 96ms 2025-11-17 11:39:55,337 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Adding replicas to map for block pool BP-655327536-127.0.1.1-1762954302619 on volume /home/zwk/hadoop/hadoop-3.3.5/tmp/dfs/data... 2025-11-17 11:39:55,337 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.BlockPoolSlice: Replica Cache file: /home/zwk/hadoop/hadoop-3.3.5/tmp/dfs/data/current/BP-655327536-127.0.1.1-1762954302619/current/replicas doesn't exist 2025-11-17 11:39:55,427 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Time to add replicas to map for block pool BP-655327536-127.0.1.1-1762954302619 on volume /home/zwk/hadoop/hadoop-3.3.5/tmp/dfs/data: 90ms 2025-11-17 11:39:55,428 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Total time to add all replicas to map for block pool BP-655327536-127.0.1.1-1762954302619: 92ms 2025-11-17 11:39:55,428 INFO org.apache.hadoop.hdfs.server.datanode.checker.ThrottledAsyncChecker: Scheduling a check for /home/zwk/hadoop/hadoop-3.3.5/tmp/dfs/data 2025-11-17 11:39:55,447 INFO org.apache.hadoop.hdfs.server.datanode.checker.DatasetVolumeChecker: Scheduled health check for volume /home/zwk/hadoop/hadoop-3.3.5/tmp/dfs/data 2025-11-17 11:39:55,479 INFO org.apache.hadoop.hdfs.server.datanode.VolumeScanner: VolumeScanner(/home/zwk/hadoop/hadoop-3.3.5/tmp/dfs/data, DS-5c441fde-88f8-4f39-a77a-075dfdc1cdf9): no suitable block pools found to scan. Waiting 1417960410 ms. 2025-11-17 11:39:55,486 WARN org.apache.hadoop.hdfs.server.datanode.DirectoryScanner: dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value above 1000 ms/sec. Assuming default value of -1 2025-11-17 11:39:55,487 INFO org.apache.hadoop.hdfs.server.datanode.DirectoryScanner: Periodic Directory Tree Verification scan starting in 3506423ms with interval of 21600000ms and throttle limit of -1ms/s 2025-11-17 11:39:55,504 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Block pool BP-655327536-127.0.1.1-1762954302619 (Datanode Uuid c229f5e4-2241-43fd-8c64-6086b77fa49a) service to localhost/127.0.0.1:9000 beginning handshake with NN 2025-11-17 11:39:55,682 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Block pool BP-655327536-127.0.1.1-1762954302619 (Datanode Uuid c229f5e4-2241-43fd-8c64-6086b77fa49a) service to localhost/127.0.0.1:9000 successfully registered with NN 2025-11-17 11:39:55,682 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: For namenode localhost/127.0.0.1:9000 using BLOCKREPORT_INTERVAL of 21600000msecs CACHEREPORT_INTERVAL of 10000msecs Initial delay: 0msecs; heartBeatInterval=3000 2025-11-17 11:39:56,204 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Successfully sent block report 0xdb29685bfe76cad with lease ID 0xf06eba0d25825652 to namenode: localhost/127.0.0.1:9000, containing 1 storage report(s), of which we sent 1. The reports had 59 total blocks and used 1 RPC(s). This took 23 msecs to generate and 224 msecs for RPC and NN processing. Got back one command: FinalizeCommand/5. 2025-11-17 11:39:56,208 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Got finalize command for block pool BP-655327536-127.0.1.1-1762954302619 2025-11-17 11:41:22,791 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-655327536-127.0.1.1-1762954302619:blk_1073741919_1102 src: /127.0.0.1:39016 dest: /127.0.0.1:9866 2025-11-17 11:41:22,834 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /127.0.0.1:39016, dest: /127.0.0.1:9866, bytes: 202, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-433486687_1, offset: 0, srvID: c229f5e4-2241-43fd-8c64-6086b77fa49a, blockid: BP-655327536-127.0.1.1-1762954302619:blk_1073741919_1102, duration(ns): 15716338 2025-11-17 11:41:22,834 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-655327536-127.0.1.1-1762954302619:blk_1073741919_1102, type=LAST_IN_PIPELINE terminating 2025-11-17 11:41:25,717 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: BlockRecoveryWorker: NameNode at localhost/127.0.0.1:9000 calls recoverBlock(BP-655327536-127.0.1.1-1762954302619:blk_1073741918_1101, targets=[DatanodeInfoWithStorage[127.0.0.1:9866,null,null]], newGenerationStamp=1103, newBlock=null, isStriped=false) 2025-11-17 11:41:25,719 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: initReplicaRecovery: blk_1073741918_1101, recoveryId=1103, replica=ReplicaWaitingToBeRecovered, blk_1073741918_1101, RWR getNumBytes() = 85 getBytesOnDisk() = 85 getVisibleLength()= -1 getVolume() = /home/zwk/hadoop/hadoop-3.3.5/tmp/dfs/data getBlockURI() = file:/home/zwk/hadoop/hadoop-3.3.5/tmp/dfs/data/current/BP-655327536-127.0.1.1-1762954302619/current/rbw/blk_1073741918 2025-11-17 11:41:25,719 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: initReplicaRecovery: changing replica state for blk_1073741918_1101 from RWR to RUR 2025-11-17 11:41:25,721 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: BlockRecoveryWorker: block=BP-655327536-127.0.1.1-1762954302619:blk_1073741918_1101 (length=0), isTruncateRecovery=false, syncList=[block:blk_1073741918_1101[numBytes=85,originalReplicaState=RWR] node:DatanodeInfoWithStorage[127.0.0.1:9866,null,null]] 2025-11-17 11:41:25,721 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: BlockRecoveryWorker: block=BP-655327536-127.0.1.1-1762954302619:blk_1073741918_1101 (length=0), bestState=RWR, newBlock=BP-655327536-127.0.1.1-1762954302619:blk_1073741918_1103 (length=85), participatingList=[block:blk_1073741918_1101[numBytes=85,originalReplicaState=RWR] node:DatanodeInfoWithStorage[127.0.0.1:9866,null,null]] 2025-11-17 11:41:25,722 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: updateReplica: BP-655327536-127.0.1.1-1762954302619:blk_1073741918_1101[numBytes=85,originalReplicaState=RWR], recoveryId=1103, length=85, replica=ReplicaUnderRecovery, blk_1073741918_1101, RUR getNumBytes() = 85 getBytesOnDisk() = 85 getVisibleLength()= -1 getVolume() = /home/zwk/hadoop/hadoop-3.3.5/tmp/dfs/data getBlockURI() = file:/home/zwk/hadoop/hadoop-3.3.5/tmp/dfs/data/current/BP-655327536-127.0.1.1-1762954302619/current/rbw/blk_1073741918 recoveryId=1103 original=ReplicaWaitingToBeRecovered, blk_1073741918_1101, RWR getNumBytes() = 85 getBytesOnDisk() = 85 getVisibleLength()= -1 getVolume() = /home/zwk/hadoop/hadoop-3.3.5/tmp/dfs/data getBlockURI() = file:/home/zwk/hadoop/hadoop-3.3.5/tmp/dfs/data/current/BP-655327536-127.0.1.1-1762954302619/current/rbw/blk_1073741918 2025-11-17 11:41:27,636 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-655327536-127.0.1.1-1762954302619:blk_1073741920_1104 src: /127.0.0.1:39028 dest: /127.0.0.1:9866 2025-11-17 11:41:28,738 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService: Scheduling blk_1073741911_1091 replica FinalizedReplica, blk_1073741911_1091, FINALIZED getNumBytes() = 202 getBytesOnDisk() = 202 getVisibleLength()= 202 getVolume() = /home/zwk/hadoop/hadoop-3.3.5/tmp/dfs/data getBlockURI() = file:/home/zwk/hadoop/hadoop-3.3.5/tmp/dfs/data/current/BP-655327536-127.0.1.1-1762954302619/current/finalized/subdir0/subdir0/blk_1073741911 for deletion 2025-11-17 11:41:28,742 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService: Deleted BP-655327536-127.0.1.1-1762954302619 blk_1073741911_1091 URI file:/home/zwk/hadoop/hadoop-3.3.5/tmp/dfs/data/current/BP-655327536-127.0.1.1-1762954302619/current/finalized/subdir0/subdir0/blk_1073741911 2025-11-17 11:41:31,695 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService: Scheduling blk_1073741918_1103 replica FinalizedReplica, blk_1073741918_1103, FINALIZED getNumBytes() = 85 getBytesOnDisk() = 85 getVisibleLength()= 85 getVolume() = /home/zwk/hadoop/hadoop-3.3.5/tmp/dfs/data getBlockURI() = file:/home/zwk/hadoop/hadoop-3.3.5/tmp/dfs/data/current/BP-655327536-127.0.1.1-1762954302619/current/finalized/subdir0/subdir0/blk_1073741918 for deletion 2025-11-17 11:41:31,696 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService: Deleted BP-655327536-127.0.1.1-1762954302619 blk_1073741918_1103 URI file:/home/zwk/hadoop/hadoop-3.3.5/tmp/dfs/data/current/BP-655327536-127.0.1.1-1762954302619/current/finalized/subdir0/subdir0/blk_1073741918 2025-11-17 11:41:33,777 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-655327536-127.0.1.1-1762954302619:blk_1073741921_1105 src: /127.0.0.1:47140 dest: /127.0.0.1:9866 2025-11-17 11:41:34,429 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-655327536-127.0.1.1-1762954302619:blk_1073741922_1106 src: /127.0.0.1:47168 dest: /127.0.0.1:9866 2025-11-17 11:49:36,751 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-655327536-127.0.1.1-1762954302619:blk_1073741923_1107 src: /127.0.0.1:53574 dest: /127.0.0.1:9866 2025-11-17 11:49:36,852 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /127.0.0.1:53574, dest: /127.0.0.1:9866, bytes: 7988, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1974153692_1, offset: 0, srvID: c229f5e4-2241-43fd-8c64-6086b77fa49a, blockid: BP-655327536-127.0.1.1-1762954302619:blk_1073741923_1107, duration(ns): 22420015 2025-11-17 11:49:36,852 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-655327536-127.0.1.1-1762954302619:blk_1073741923_1107, type=LAST_IN_PIPELINE terminating 2025-11-17 11:56:28,899 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-655327536-127.0.1.1-1762954302619:blk_1073741924_1108 src: /127.0.0.1:55434 dest: /127.0.0.1:9866 2025-11-17 11:56:28,909 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /127.0.0.1:55434, dest: /127.0.0.1:9866, bytes: 5576, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-433486687_1, offset: 0, srvID: c229f5e4-2241-43fd-8c64-6086b77fa49a, blockid: BP-655327536-127.0.1.1-1762954302619:blk_1073741924_1108, duration(ns): 7727488 2025-11-17 11:56:28,909 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-655327536-127.0.1.1-1762954302619:blk_1073741924_1108, type=LAST_IN_PIPELINE terminating 2025-11-17 11:56:29,162 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-655327536-127.0.1.1-1762954302619:blk_1073741925_1109 src: /127.0.0.1:55456 dest: /127.0.0.1:9866 2025-11-17 11:56:29,172 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /127.0.0.1:55456, dest: /127.0.0.1:9866, bytes: 5855, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-433486687_1, offset: 0, srvID: c229f5e4-2241-43fd-8c64-6086b77fa49a, blockid: BP-655327536-127.0.1.1-1762954302619:blk_1073741925_1109, duration(ns): 6272953 2025-11-17 11:56:29,174 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-655327536-127.0.1.1-1762954302619:blk_1073741925_1109, type=LAST_IN_PIPELINE terminating 2025-11-17 11:56:29,242 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-655327536-127.0.1.1-1762954302619:blk_1073741926_1110 src: /127.0.0.1:55462 dest: /127.0.0.1:9866 2025-11-17 11:56:29,260 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /127.0.0.1:55462, dest: /127.0.0.1:9866, bytes: 5055, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-433486687_1, offset: 0, srvID: c229f5e4-2241-43fd-8c64-6086b77fa49a, blockid: BP-655327536-127.0.1.1-1762954302619:blk_1073741926_1110, duration(ns): 14400418 2025-11-17 11:56:29,263 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-655327536-127.0.1.1-1762954302619:blk_1073741926_1110, type=LAST_IN_PIPELINE terminating 2025-11-17 11:56:33,524 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /127.0.0.1:39028, dest: /127.0.0.1:9866, bytes: 66426, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-433486687_1, offset: 0, srvID: c229f5e4-2241-43fd-8c64-6086b77fa49a, blockid: BP-655327536-127.0.1.1-1762954302619:blk_1073741920_1104, duration(ns): 905884789428 2025-11-17 11:56:33,524 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-655327536-127.0.1.1-1762954302619:blk_1073741920_1104, type=LAST_IN_PIPELINE terminating这其中有什么信息
11-18
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符  | 博主筛选后可见
 
红包 添加红包
表情包 插入表情
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值