Skip to main content

Hortonworks Distribution Platform(HDP)

Following Manually Install JuiceFS Java Client, and then modify the configuration at Ambari:

  • Hadoop

    Modify core-site.xml at HDFS interface, following Parameter Configuration.

  • MapReduce2

    Modify mapreduce.application.classpath at MapReduce2 interface, append :/usr/hdp/${hdp.version}/hadoop/lib/juicefs-hadoop.jar at the last(don\'t replace the variable)

  • Hive

    Modify hive.metastore.warehouse.dir for default location of table creation at Hive interface(optional):

    jfs://your-jfs-name/your-warehouse-dir

    If you have the Ranger service, please append jfs at the last of ranger.plugin.hive.urlauth.filesystem.schemes:

    ranger.plugin.hive.urlauth.filesystem.schemes=hdfs:,file:,wasb:,adl:,jfs:
  • Druid

    Change the directory address at Driud interface (Maybe you need to create the directory manually due to permission limit) :

    "druid.storage.storageDirectory": "jfs://your-jfs-name/apps/druid/warehouse"
    "druid.indexer.logs.directory": "jfs://your-jfs-name/user/druid/logs"
  • HBase

    Modify these params at HBase

    hbase.rootdir=jfs://your-jfs-name/hbase
    hbase.wal.dir=hdfs://your-hdfs-uri/hbase-wal

    Delete znode(default /hbase) zookeeper.znode.parent

    Notice:this operation will empty HBase data

  • Sqoop

    When import data into Hive by Sqoop, Sqoop should imports data into target-dir first, and then loads into the Hive table with hive load, so you need to modify target-dir when using Sqoop.

    1.4.6

    For version 1.4.6, You need modify fs to update the default filesystem. Please copy mapreduce.tar.gz to the same path from HDFS to JuiceFS, defalt path is /hdp/apps/${hdp.version}/mapreduce/mapreduce.tar.gz.

    sqoop import \
    -fs jfs://your-jfs-name/ \
    --target-dir jfs://your-jfs-name/tmp/your-dir

    1.4.7

    sqoop import \
    --target-dir jfs://your-jfs-name/tmp/your-dir

Finally, restart the service to take effect changes.