`
standalone
  • 浏览: 596546 次
  • 性别: Icon_minigender_1
  • 来自: 上海
社区版块
存档分类
最新评论

Using the libjars option with Hadoop

 
阅读更多
As I have said in my last post, I was developing a hbase based mapreduce application. But one damn thing is the hadoop cluster managed by our system admin has no hbase jars in its classpath...

So I have to figure out how to provide jars needed by hbase to hadoop. I know "hadoop jar" command has an option "-libjars" which allow you to provide third party jars needed by your mapper or reducer. I did tried it. But it still says class HBaseConfiguration cannot be found. I tried many times without a success.

I got stuck at this...

Then a long weekend.

Today is monday. I continue work on this. I should mention that I have read the articles like these:

http://blog.cloudera.com/blog/2011/01/how-to-include-third-party-libraries-in-your-map-reduce-job/

http://grepalex.com/2013/02/25/hadoop-libjars/

But still failed. Finally I add the jars to HADOOP_CLASSPATH and set HADOOP_USER_CLASSPATH_FIRST true, then rerun the job, no errors any more!

分享到:
评论

相关推荐

    Scaling Big Data with Hadoop and Solr

    Starting with the basics of Apache Hadoop and Solr, this book then dives into advanced topics of optimizing search with some interesting real-world use cases and sample Java code.

    Modern Big Data Processing with Hadoop

    to make the information more accessible to the users.This book empowers you to build such solutions with relative ease with the help of Apache Hadoop, along with a host of other Big Data tools. ...

    Data Analytics with Hadoop: An Introduction for Data Scientists

    operations, or software development usually associated with distributed computing, you’ll focus on particular analyses you can build, the data warehousing techniques that Hadoop provides, and higher...

    Deep Learning with Hadoop

    Starting with understanding what deep learning is and what the various models associated with deep learning are, this book will then show you how to set up the Hadoop environment for deep learning....

    Data Analytics with Hadoop(O'Reilly,2016)

    operations, or software development usually associated with distributed computing, you’ll focus on particular analyses you can build, the data warehousing techniques that Hadoop provides, and higher...

    Data Algorithms Recipes for Scaling Up with Hadoop and Spark epub

    Data Algorithms Recipes for Scaling Up with Hadoop and Spark 英文epub 本资源转载自网络,如有侵权,请联系上传者或csdn删除 本资源转载自网络,如有侵权,请联系上传者或csdn删除

    Modern Big Data Processing with Hadoop-Packt Publishing(2018)

    Apache Hadoop, along with a host of other big data tools, empowers you to build such solutions with relative ease. This book lists some unique ideas and techniques that enable you to conquer ...

    Big Data Processing With Hadoop 1st Edition

    Big Data Processing With Hadoop is an essential reference source that discusses possible solutions for millions of users working with a variety of data applications, who expect fast turnaround ...

    Big Data Analytics with Hadoop 3

    Explore big data concepts, platforms, analytics, and their applications using the power of Hadoop 3 Apache Hadoop is the most popular platform for big data processing, and can be combined with a host ...

    Big Data Processing With Hadoop

    Big Data Processing With Hadoop

    Data Analytics with Hadoop

    Data Analytics with Hadoop,数据处理,数据分析,数据挖掘等方面的经典教材

    Scaling.Big.Data.with.Hadoop.and.Solr.2nd.Edition.1783553391

    Starting with the basics of Apache Hadoop and Solr, the book covers advanced topics of optimizing search with some interesting real-world use cases and sample Java code. This is a step-by-step guide...

    Deep Learning with Hadoop epub

    Deep Learning with Hadoop 英文epub 本资源转载自网络,如有侵权,请联系上传者或csdn删除 查看此书详细信息请在美国亚马逊官网搜索此书

    Hadoop: The Definitive Guide

    Hadoop: The Definitive Guide is a comprehensive resource for using Hadoop to build reliable, scalable, distributed systems. Programmers will find details for analyzing large datasets with Hadoop, and...

    Data Analytics with Hadoop 无水印pdf

    Data Analytics with Hadoop 英文无水印pdf pdf所有页面使用FoxitReader和PDF-XChangeViewer测试都可以打开 本资源转载自网络,如有侵权,请联系上传者或csdn删除 本资源转载自网络,如有侵权,请联系上传者或...

    Deep learning with Hadoop : build, implement and scale distributed d l models

    deep neural networks with Hadoop for optimal performance. Starting with understanding what deep learning is, and what the various models associated with deep neural networks are, this book will then ...

    Hadoop The Definitive Guide PDF

    书名:Hadoop The Definitive Guide 语言:英文 The rest of this book is organized as follows. Chapter 2 provides an introduction to MapReduce. Chapter 3 looks at Hadoop filesystems, and in particular ...

Global site tag (gtag.js) - Google Analytics