Hive(Java)のコードまたはクラスをちょっとだけ変えてみる

例：https://issues.apache.org/jira/secure/attachment/12749206/HIVE-11498.003.patch

1) どのjarかわからない場合、全部探してみる
find -L /usr/hdp/current/hive-client/ -type f -name '*.jar' -print0 | xargs -0 -n1 -I {} bash -c "less {} | grep -w Driver && echo {}"
# NOTE: シンボリックリンクのため最後の"/"を忘れない(または-L)
…
-rw----     2.0 fat    64666 bl defN 15-Sep-30 19:09 org/apache/hadoop/hive/ql/Driver.class
/usr/hdp/current/hive-client/lib/hive-exec-1.2.1.2.3.2.0-2950.jar

または、
grep SymbolicInputFormat -l /usr/hdp/current/hive-client/lib/*
[root@node2 ~]# less /usr/hdp/current/hive-client/lib/hive-exec-1.2.1000.2.4.2.0-258.jar | grep SymbolicInputFormat

-rw----     2.0 fat     4917 bl defN 16-Apr-25 06:49 org/apache/hadoop/hive/ql/io/SymbolicInputFormat.class

2) バックアップを作る
cp -p /usr/hdp/current/hive-client/lib/hive-exec-1.2.1.2.3.2.0-2950.jar /tmp/hive-exec-1.2.1.2.3.2.0-2950.jar
# NOTE: バックアップはクラスパス内には作らない

3) ソースコードをダウンロードする
mkdir ~/workspace; cd ~/workspace
mkdir -p org/apache/hadoop/hive/ql
wget https://raw.githubusercontent.com/apache/hive/branch-1.2/ql/src/java/org/apache/hadoop/hive/ql/Driver.java -O org/apache/hadoop/hive/ql/Driver.java

4) 確認または編集(viかpatchコマンド）
vi Driver.java

5) クラスパスを取得（jcmd PID VM.system_properties | grep ^java.class.path などでも可）：
eval "export `cat /proc/$(cat /var/run/hive/hive-server.pid)/environ | tr '\0' '\n' | grep ^CLASSPATH`"

eval "export `strings /proc/$(cat /var/run/hive/hive-server.pid)/environ | grep ^CLASSPATH`"

sudo -u oozie /usr/jdk64/jdk1.8.0_112/bin/jcmd `cat /var/run/oozie/oozie.pid` VM.system_properties | grep java.class.path

export CLASSPATH=$(lsof -p `cat /var/run/oozie/oozie.pid` | grep -oE '/.+\.jar$' | tr '\n' ':')

6) コンパイル
source /etc/hadoop/conf/hadoop-env.sh
$JAVA_HOME/bin/javac org/apache/hadoop/hive/ql/Driver.java

7) Jarを更新する
$JAVA_HOME/bin/jar uf /usr/hdp/current/hive-client/lib/hive-exec-1.2.1.2.3.2.0-2950.jar org/apache/hadoop/hive/ql/Driver*class

8) 確認
less /usr/hdp/current/hive-client/lib/hive-exec-1.2.1.2.3.2.0-2950.jar | grep '/Driver'


ちなみにADD JAR用に.jarファイル(dummy.jar)を作るには
eval "export `cat /proc/$(cat /var/run/hive/hive.pid)/environ | tr '\0' '\n' | grep ^CLASSPATH`
$JAVA_HOME/bin/javac dummy/*.java
$JAVA_HOME/bin/jar cvf dummy.jar dummy/*.class
# 必要なテーブルを作成後
hive -e 'ADD JAR /root/dummy.jar;insert into dummy select * from dummy2'

Rangerの場合：
[root@node3 classes]# pwd
/usr/hdp/2.5.5.0-157/ranger-admin/ews/webapp/WEB-INF/classes
[root@node3 classes]# export CLASSPATH="`find /usr/hdp/current/ranger-admin/ -name '*.jar' | tr '\n' ':'`"
[root@node3 classes]# /usr/jdk64/jdk1.8.0_112/bin/javac org/apache/ranger/biz/UserMgr.java
[root@node3 classes]# sudo -u ranger /usr/bin/ranger-admin restart

Ambariから Hive用のJDBCドライバーを追加・更新する

On Ambari Server

1) Download the latest mysql-connector-java-xxxx.jar and copy into this server's /usr/share/java/ 

2) ln -sf /usr/share/java/mysql-connector-java-xxxx.jar /usr/share/java/mysql-connector-java.jar

3) ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java.jar 

4) find /var/lib/ambari-server/resources -name 'mysql-*.jar' -ls Above is to make sure it's updated.

On HiveServer2/Metastore Server

1) find / -name 'mysql-*.jar' -ls 

2) remove old mysql-connector-xxx.jar from Agent's tmp directory and /usr/hdb/<version>/hive/lib 

3) remove old mysql-jdbc-driver.jar from Agent's cache directory 

4) Replace /usr/hdp/<version>/hadoop/lib/mysql-connector-java.jar with newer version if exists.

5) Restart ambari-agent 

6) Restart Hive (hiveserver2/metastore) from Ambari 

7) Run find command to make sure the version is correct by checking the file size

JDBCドライバーバージョンを 素早く確認する

A) MySQL:
$ zipgrep 'Bundle-Version' /usr/hdp/current/hive-client/lib/mysql-connector*.jar
$ zipgrep 'Bundle-Version' /usr/share/java/mysql-connector-java.jar

出力例：
META-INF/MANIFEST.MF:Bundle-Version: 5.1.39

サーバ側:
ただ単に、mysqlコマンドを使うか
SHOW VARIABLES LIKE "%version%";

B) PostgreSQL
$ zipgrep 'Bundle-Version' /usr/hdp/current/hive-client/lib/postgresql*.jar
$ zipgrep 'Bundle-Version' /usr/share/java/postgresql-jdbc.jar

出力例：
META-INF/MANIFEST.MF:Bundle-Version: 9.4.1208.jre7

古いバージョンだと何も出ない模様、その場合は:
$ unzip -c /usr/share/java/postgresql-jdbc.jar 'org/postgresql/Driver.class' | grep -oP 'PostgreSQL.+?\)'
PostgreSQL 9.0 JDBC4 (build 801)

サーバ側 (psql)：
SELECT version();

C) Oracle
$ zipgrep 'Implementation-Version' /usr/share/java/ojdbc6.jar
META-INF/MANIFEST.MF:Implementation-Version: 11.2.0.4.0

サーバ側（sqlplus）:
SELECT * FROM v$version;

NOTE: Oracle JDBC と server version:
http://www.oracle.com/technetwork/database/enterprise-edition/jdbc-faq-090281.html#01_02