你好,游客 登录 注册 搜索
背景:
阅读新闻

Ubuntu下Hadoop环境的配置

[日期:2012-11-21] 来源:Linux社区  作者:SpeedMe [字体: ]

6、初始化name node节点
Hadoop@ecy-geek:/usr/local/hadoop/bin$ ./hadoop namenode -format
09/10/31 23:30:10 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:  host = ecy-geek/127.0.1.1
STARTUP_MSG:  args = [-format]
STARTUP_MSG:  version = 0.20.1
STARTUP_MSG:  build = http://svn.apache.org/repos/asf/hadoop/common/tags/release-0.20.1-rc1 -r 810220; compiled by 'oom' on Tue Sep  1 20:55:56 UTC 2009
************************************************************/
Re-format filesystem in /usr/local/hadoop-datastore/hadoop-hadoop/dfs/name ? (Y or N) y
Format aborted in /usr/local/hadoop-datastore/hadoop-hadoop/dfs/name
09/10/31 23:30:16 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at ecy-geek/127.0.1.1
************************************************************/
    7、运行Hadoop
hadoop@ecy-geek:/usr/local/hadoop/bin$ ./start-all.sh
starting namenode, logging to /usr/local/hadoop/bin/../logs/hadoop-hadoop-namenode-ecy-geek.out
localhost: starting datanode, logging to /usr/local/hadoop/bin/../logs/hadoop-hadoop-datanode-ecy-geek.out
localhost: starting secondarynamenode, logging to /usr/local/hadoop/bin/../logs/hadoop-hadoop-secondarynamenode-ecy-geek.out
starting jobtracker, logging to /usr/local/hadoop/bin/../logs/hadoop-hadoop-jobtracker-ecy-geek.out
localhost: starting tasktracker, logging to /usr/local/hadoop/bin/../logs/hadoop-hadoop-tasktracker-ecy-geek.out
    可见namenode,datanode,secondarynamenode,jobtracker,tasktracker都运行起来了,使用jps可显示相关信息。
hadoop@ecy-geek:/usr/local/hadoop/bin$ jps
21581 NameNode
21975 SecondaryNameNode
22238 TaskTracker
22477 Jps
22053 JobTracker
21777 DataNode
    呵呵,到这里就差不多,Hadoop提供了方便的Web UI以查看相关信息,地址分别如下:
http://localhost:50030/ - web UI for MapReduce job tracker(s)
http://localhost:50060/ - web UI for task tracker(s)
http://localhost:50070/ - web UI for HDFS name node(s)
      接着就可以运行MapReduce job了,也可以通过DSF Shell来操作分布式文件系统,可惜是单机的,如果有许多台机器就壮观了。

其中我遇到一个问题

hadoop@Ubuntu:/usr/local/hadoop$ bin/start-all.sh
mkdir: cannot create directory `/usr/local/hadoop/bin/../logs': Permission denied
starting namenode, logging to /usr/local/hadoop/bin/../logs/hadoop-hadoop-namenode-ubuntu.out
/usr/local/hadoop/bin/hadoop-daemon.sh: line 117: /usr/local/hadoop/bin/../logs/hadoop-hadoop-namenode-ubuntu.out: No such file or directory

解决:修改hadoop文件夹的权限,保证hadoop用户能正常访问其中的文件sudo chown -hR hadoop /usr/local/hadoop

主要是因为hadoop用户没权限。

linux
本文评论   查看全部评论 (0)
表情: 表情 姓名: 字数

       

评论声明
  • 尊重网上道德,遵守中华人民共和国的各项有关法律法规
  • 承担一切因您的行为而直接或间接导致的民事或刑事法律责任
  • 本站管理人员有权保留或删除其管辖留言中的任意内容
  • 本站有权在网站内转载或引用您的评论
  • 参与本评论即表明您已经阅读并接受上述条款