hadoop hdfs-default.xml配置文件

本文深入探讨了Hadoop HDFS-default.xml配置文件,详细解析了各个配置选项的作用和最佳实践,帮助读者理解HDFS的底层工作原理,提升大数据存储和运维效率。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

namevaluedescription
dfs.namenode.logging.levelinfoThe logging level for dfs namenode. Other values are "dir"(trac e namespace mutations), "block"(trace block under/over replications and block creations/deletions), or "all".
dfs.secondary.http.address0.0.0.0:50090The secondary namenode http server address and port. If the port is 0 then the server will start on a free port.
dfs.datanode.address0.0.0.0:50010The address where the datanode server will listen to. If the port is 0 then the server will start on a free port.
dfs.datanode.http.address0.0.0.0:50075The datanode http server address and port. If the port is 0 then the server will start on a free port.
dfs.datanode.ipc.address0.0.0.0:50020The datanode ipc server address and port. If the port is 0 then the server will start on a free port.
dfs.datanode.handler.count3The number of server threads for the datanode.
dfs.http.address0.0.0.0:50070The address and the base port where the dfs namenode web ui will listen on. If the port is 0 then the server will start on a free port.
dfs.https.enablefalseDecide if HTTPS(SSL) is supported on HDFS
dfs.https.need.client.authfalseWhether SSL client certificate authentication is required
dfs.https.server.keystore.resourcessl-server.xmlResource file from which ssl server keystore information will be extracted
dfs.https.client.keystore.resourcessl-client.xmlResource file from which ssl client keystore information will be extracted
dfs.datanode.https.address0.0.0.0:50475 
dfs.https.address0.0.0.0:50470 
dfs.datanode.dns.interfacedefaultThe name of the Network Interface from which a data node should report its IP address.
dfs.datanode.dns.nameserverdefaultThe host name or IP address of the name server (DNS) which a DataNode should use to determine the host name used by the NameNode for communication and display purposes.
dfs.replication.considerLoadtrueDecide if chooseTarget considers the target's load or not
dfs.default.chunk.view.size32768The number of bytes to view for a file on the browser.
dfs.datanode.du.reserved0Reserved space in bytes per volume. Always leave this much space free for non dfs use.
dfs.name.dir${hadoop.tmp.dir}/dfs/nameDetermines where on the local filesystem the DFS name node should store the name table(fsimage). If this is a comma-delimited list of directories then the name table is replicated in all of the directories, for redundancy.
dfs.name.edits.dir${dfs.name.dir}Determines where on the local filesystem the DFS name node should store the transaction (edits) file. If this is a comma-delimited list of directories then the transaction file is replicated in all of the directories, for redundancy. Default value is same as dfs.name.dir
dfs.web.ugiwebuser,webgroupThe user account used by the web interface. Syntax: USERNAME,GROUP1,GROUP2, ...
dfs.permissionstrueIf "true", enable permission checking in HDFS. If "false", permission checking is turned off, but all other behavior is unchanged. Switching from one parameter value to the other does not change the mode, owner or group of files or directories.
dfs.permissions.supergroupsupergroupThe name of the group of super-users.
dfs.block.access.token.enablefalseIf "true", access tokens are used as capabilities for accessing datanodes. If "false", no access tokens are checked on accessing datanodes.
dfs.block.access.key.update.interval600Interval in minutes at which namenode updates its access keys.
dfs.block.access.token.lifetime600The lifetime of access tokens in minutes.
dfs.data.dir${hadoop.tmp.dir}/dfs/dataDetermines where on the local filesystem an DFS data node should store its blocks. If this is a comma-delimited list of directories, then data will be stored in all named directories, typically on different devices. Directories that do not exist are ignored.
dfs.datanode.data.dir.perm755Permissions for the directories on on the local filesystem where the DFS data node store its blocks. The permissions can either be octal or symbolic.
dfs.replication3Default block replication. The actual number of replications can be specified when the file is created. The default is used if replication is not specified in create time.
dfs.replication.max512Maximal block replication.
dfs.replication.min1Minimal block replication.
dfs.block.size67108864The default block size for new files.
dfs.df.interval60000Disk usage statistics refresh interval in msec.
dfs.client.block.write.retries3The number of retries for writing blocks to the data nodes, before we signal failure to the application.
dfs.blockreport.intervalMsec3600000Determines block reporting interval in milliseconds.
dfs.blockreport.initialDelay0Delay for first block report in seconds.
dfs.heartbeat.interval3Determines datanode heartbeat interval in seconds.
dfs.namenode.handler.count10The number of server threads for the namenode.
dfs.safemode.threshold.pct0.999fSpecifies the percentage of blocks that should satisfy the minimal replication requirement defined by dfs.replication.min. Values less than or equal to 0 mean not to start in safe mode. Values greater than 1 will make safe mode permanent.
dfs.safemode.extension30000Determines extension of safe mode in milliseconds after the threshold level is reached.
dfs.balance.bandwidthPerSec1048576Specifies the maximum amount of bandwidth that each datanode can utilize for the balancing purpose in term of the number of bytes per second.
dfs.hosts Names a file that contains a list of hosts that are permitted to connect to the namenode. The full pathname of the file must be specified. If the value is empty, all hosts are permitted.
dfs.hosts.exclude Names a file that contains a list of hosts that are not permitted to connect to the namenode. The full pathname of the file must be specified. If the value is empty, no hosts are excluded.
dfs.max.objects0The maximum number of files, directories and blocks dfs supports. A value of zero indicates no limit to the number of objects that dfs supports.
dfs.namenode.decommission.interval30Namenode periodicity in seconds to check if decommission is complete.
dfs.namenode.decommission.nodes.per.interval5The number of nodes namenode checks if decommission is complete in each dfs.namenode.decommission.interval.
dfs.replication.interval3The periodicity in seconds with which the namenode computes repliaction work for datanodes.
dfs.access.time.precision3600000The access time for HDFS file is precise upto this value. The default value is 1 hour. Setting a value of 0 disables access times for HDFS.
dfs.support.appendfalseDoes HDFS allow appends to files? This is currently set to false because there are bugs in the "append code" and is not supported in any prodction cluster.
dfs.namenode.delegation.key.update-interval86400000The update interval for master key for delegation tokens in the namenode in milliseconds.
dfs.namenode.delegation.token.max-lifetime604800000The maximum lifetime in milliseconds for which a delegation token is valid.
dfs.namenode.delegation.token.renew-interval86400000The renewal interval for delegation token in milliseconds.
dfs.datanode.failed.volumes.tolerated0The number of volumes that are allowed to fail before a datanode stops offering service. By default any volume failure will cause a datanode to shutdown.
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值