参考：
https://hadoop-and-hdp.blogspot.in/2017/09/ambari-240hdp-search-solrhdp-250.html
https://community.hortonworks.com/articles/15159/securing-solr-collections-with-ranger-kerberos.html Kerberosパートは必要ない？もう古い？
https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5.0+Installation#ApacheRanger0.5.0Installation-EnablingRangerSolrPlugin
https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.2/bk_security/content/solr_service.html
https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.0/bk_solr-search-installation/content/ch_hdp-search-install-ambari.html
http://public-repo-1.hortonworks.com/HDP-SOLR-2.6.1-100/repos/centos6/HDP-SOLR-2.6-100-centos6.tar.gz
http://public-repo-1.hortonworks.com/HDP-SOLR/hdp-solr-ambari-mp/solr-service-mpack-2.2.9.tar.gz
cp /usr/share/java/mysql-connector-java.jar /usr/hdp/`hdp-select versions | tail -1`/ranger-solr-plugin/lib
# コンフィグライルへのパスは何回も使うので_fに格納
_f="/usr/hdp/`hdp-select versions | tail -1`/ranger-solr-plugin/install.properties"
# 現在の値を確認（後で同じコマンドで確認）
grep -E '^(SQL_CONNECTOR_JAR|COMPONENT_INSTALL_DIR_NAME|POLICY_MGR_URL|REPOSITORY_NAME)' $_f
COMPONENT_INSTALL_DIR_NAME=/opt/solr/server
POLICY_MGR_URL=
SQL_CONNECTOR_JAR=/usr/share/java/mysql-connector-java.jar
REPOSITORY_NAME=
# 念のためバックアップ
cp -p $_f $_f.bak
# 使用する変数の指定
_RANGER_HOST="node13.localdomain"
_ZOOKEEPER_HOST="node13.localdomain"
_CLUSTER_NAME="ubu02c11"
# 設定変更（太字の箇所は変更する場合あり）
sed -i -e 's@^COMPONENT_INSTALL_DIR_NAME=.*@COMPONENT_INSTALL_DIR_NAME=/opt/lucidworks-hdpsearch/solr/server@' $_f
sed -i -e 's@^POLICY_MGR_URL=.*@POLICY_MGR_URL=http://'$_RANGER_HOST':6080@' $_f
sed -i -e 's@^REPOSITORY_NAME=.*@REPOSITORY_NAME='$_CLUSTER_NAME'_solr@' $_f
#sed -i -e 's@^SQL_CONNECTOR_JAR=.*@SQL_CONNECTOR_JAR=/usr/share/java/mysql-connector-java.jar@' $_f
# 念のためタイムアウトを増加
sed -i 's/(sleep 5)/(sleep 30)/g' /opt/lucidworks-hdpsearch/solr/bin/solr
service solr restart
# Mpackの場合はAmbariからリスタート
Note:"/solr"が必要かどうかは不明。HWXとApacheで記述が異なる
# ステータスの確認
/opt/lucidworks-hdpsearch/solr/bin/solr status
Found 1 Solr nodes:
Solr process 2216 running on port 8983
INFO - 2017-09-06 09:37:14.907; org.apache.solr.util.SolrCLI; Set HttpClientConfigurer from: org.apache.solr.client.solrj.impl.Krb5HttpClientConfigurer
INFO - 2017-09-06 09:37:15.116; org.apache.solr.client.solrj.impl.Krb5HttpClientConfigurer; Setting up SPNego auth with config: /etc/solr/conf/solr_server_jaas.conf
{
"solr_home":"/etc/solr/data_dir",
"version":"5.5.2 8e5d40b22a3968df065dfc078ef81cbb031f0e4a - sarowe - 2016-06-21 11:44:11",
"startTime":"2017-09-06T09:36:49.46Z",
"uptime":"0 days, 0 hours, 0 minutes, 26 seconds",
"memory":"84.6 MB (%17.3) of 490.7 MB",
"cloud":{
"ZooKeeper":"node13.localdomain:2181/solr",
"liveNodes":"1",
"collections":"1"}}
# Ranger Solr pluginのPolicyがダウンロドできるか確認
su - solr
kinit -kt /etc/security/keytabs/solr.service.keytab solr/sandbox.hortonworks.com
curl -v -u amb_ranger_admin "http://sandbox.hortonworks.com:6080/service/plugins/secure/policies/download/Sandbox_solr"
curl -v --negotiate -u: "http://sandbox.hortonworks.com:6080/service/plugins/secure/policies/download/Sandbox_solr"
1. collection1_shard1_replica1: org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: Index dir 'hdfs://node12.localdomain:8020/solr/collection1/core_node1/data/index/' of core 'collection1_shard1_replica1' is already locked. The most likely cause is another Solr server (or another solr core in this server) also configured to use this directory; other possible causes may be specific to lockType: hdfs
https://issues.apache.org/jira/browse/SOLR-8335
hdfs dfs -ls -R /solr/ | grep write.lock
-rw-r--r-- 3 solr hdfs 0 2017-08-31 08:26 hdfs://node12.localdomain:8020/solr/collection1/core_node1/data/index/write.lock
-rw-r--r-- 3 solr hdfs 0 2017-08-31 08:26 hdfs://node12.localdomain:8020/solr/collection1/core_node2/data/index/write.lock
2. /opt/lucidworks-hdpsearch/solr/server/scripts/cloud-scripts/zkcli.sh -zkhost node13.localdomain:2181 -cmd makepath /solr
Exception in thread "main" org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /solr
at org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
at org.apache.solr.common.cloud.SolrZkClient$10.execute(SolrZkClient.java:501)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:498)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:455)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:442)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:398)
at org.apache.solr.cloud.ZkCLI.main(ZkCLI.java:258)
3. service solr start
/opt/solr not found! Please check the SOLR_INSTALL_DIR setting in your /etc/init.d/solr script.
/etc/default/solr.in.sh not found! Please check the SOLR_ENV setting in your /etc/init.d/solr script.
ln -s /opt/lucidworks-hdpsearch/solr /opt/solr
ln -s /opt/lucidworks-hdpsearch/solr/bin/solr.in.sh /etc/default/solr.in.sh
4. org.apache.ranger.authorization.hadoop.utils.RangerCredentialProvider (RangerCredentialProvider.java:72) - Unable to get the Credential Provider from the Configuration
java.lang.IllegalArgumentException: The value of property hadoop.security.credential.provider.path must not be null
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:92)
at org.apache.hadoop.conf.Configuration.set(Configuration.java:1010)
at org.apache.hadoop.conf.Configuration.set(Configuration.java:991)
at org.apache.ranger.authorization.hadoop.utils.RangerCredentialProvider.getCredentialProviders(RangerCredentialProvider.java:68)
...
/usr/hdp/2.6.0.3-8/ranger-solr-plugin/install/conf.templates/enable/ranger-policymgr-ssl.xml
/opt/lucidworks-hdpsearch/solr/server/solr-webapp/webapp/WEB-INF/classes/ranger-policymgr-ssl.xml
ln -s /opt/lucidworks-hdpsearch/solr/server/solr-webapp/webapp/WEB-INF/classes/ranger-policymgr-ssl.xml /etc/solr/conf/ranger-policymgr-ssl.xml
5. SLF4J: Failed toString() invocation on an object of type [org.apache.solr.servlet.HttpSolrCall$2]
java.lang.NullPointerException
at org.apache.solr.servlet.HttpSolrCall$2.toString(HttpSolrCall.java:1001)
at org.slf4j.helpers.MessageFormatter.safeObjectAppend(MessageFormatter.java:305)
at org.slf4j.helpers.MessageFormatter.deeplyAppendParameter(MessageFormatter.java:277)
at org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:231)
at org.slf4j.helpers.MessageFormatter.format(MessageFormatter.java:152)
at org.slf4j.impl.Log4jLoggerAdapter.info(Log4jLoggerAdapter.java:345)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:431)
...
https://issues.apache.org/jira/browse/SOLR-10630
https://issues.apache.org/jira/browse/RANGER-1446
" | grep -v ambari-infra"を追加する
そもそもmpackインストール後、security.jsonが変更できない（Ambariのデフォルトに書き換えられる）
Pluginのインストール先：/usr/hdp/2.6.0.3-8/ranger-solr-plugin
不親切にもinstall.propertiesを自分で設定する必要あり
https://issues.apache.org/jira/browse/RANGER-1446
https://issues.apache.org/jira/browse/RANGER-1658
HDP 2.6.3で修正済みの模様
Sandboxにmpackをインストールしようとすると、
ambari-server install-mpack --mpack=/tmp/solr-service-mpack-2.2.9.tar.gz --verbose
...
INFO: Loading properties from /etc/ambari-server/conf/ambari.properties
source:/var/lib/ambari-server/resources/mpacks/solr-ambari-mpack-2.2.9/common-services/SOLR/5.5.4
link_name:/var/lib/ambari-server/resources/common-services/SOLR/5.5.4
Traceback (most recent call last):
File "/usr/sbin/ambari-server.py", line 941, in <module>
... (snip)...
sudo.symlink(src_path, dest_link)
File "/usr/lib/python2.6/site-packages/resource_management/core/sudo.py", line 125, in symlink
os.symlink(source, link_name)
OSError: [Errno 17] File exists
または、
ERROR: Management pack solr-ambari-mpack-2.2.N already installed!
ERROR: Exiting with exit code -1.
REASON: Management pack solr-ambari-mpack-2.2.N already installed!
find /var/lib/ambari-server -type l -ls | grep SOLR
#rm -rf /var/lib/ambari-server/resources/mpacks/solr-ambari-mpack*
#rm /var/lib/ambari-server/resources/common-services/SOLR/5.5.2.2.*
# 多分これだけで大丈夫
rm /var/lib/ambari-server/resources/stacks/HDP/2.4/services/SOLR
#rm -rf /var/lib/ambari-server/resources/extensions/SOLR/*
NOTE: ambari-server backup でバックアップが取れる！
/etc/solr/conf/solr_server_jaas.conf
log4j.logger.org.apache.ranger.authorization.solr.authorizer.RangerSolrAuthorizer=WARN
https://hadoop-and-hdp.blogspot.in/2017/09/ambari-240hdp-search-solrhdp-250.html
https://community.hortonworks.com/articles/15159/securing-solr-collections-with-ranger-kerberos.html Kerberosパートは必要ない？もう古い？
https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5.0+Installation#ApacheRanger0.5.0Installation-EnablingRangerSolrPlugin
https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.0/bk_solr-search-installation/content/ch_hdp-search-install-ambari.html
http://public-repo-1.hortonworks.com/HDP-SOLR-2.6.1-100/repos/centos6/HDP-SOLR-2.6-100-centos6.tar.gz
http://public-repo-1.hortonworks.com/HDP-SOLR/hdp-solr-ambari-mp/solr-service-mpack-2.2.9.tar.gz
Pluginのインストール(mpackを使用していない場合）
yum install ranger_*-solr-pluginPluginの設定(mpack 2.2.5でも必要）
# 古いRangerでDBへのAuditを使用している場合のみcp /usr/share/java/mysql-connector-java.jar /usr/hdp/`hdp-select versions | tail -1`/ranger-solr-plugin/lib
# コンフィグライルへのパスは何回も使うので_fに格納
_f="/usr/hdp/`hdp-select versions | tail -1`/ranger-solr-plugin/install.properties"
# 現在の値を確認（後で同じコマンドで確認）
grep -E '^(SQL_CONNECTOR_JAR|COMPONENT_INSTALL_DIR_NAME|POLICY_MGR_URL|REPOSITORY_NAME)' $_f
COMPONENT_INSTALL_DIR_NAME=/opt/solr/server
POLICY_MGR_URL=
SQL_CONNECTOR_JAR=/usr/share/java/mysql-connector-java.jar
REPOSITORY_NAME=
# 念のためバックアップ
cp -p $_f $_f.bak
# 使用する変数の指定
_RANGER_HOST="node13.localdomain"
_ZOOKEEPER_HOST="node13.localdomain"
_CLUSTER_NAME="ubu02c11"
# 設定変更（太字の箇所は変更する場合あり）
sed -i -e 's@^COMPONENT_INSTALL_DIR_NAME=.*@COMPONENT_INSTALL_DIR_NAME=/opt/lucidworks-hdpsearch/solr/server@' $_f
sed -i -e 's@^POLICY_MGR_URL=.*@POLICY_MGR_URL=http://'$_RANGER_HOST':6080@' $_f
sed -i -e 's@^REPOSITORY_NAME=.*@REPOSITORY_NAME='$_CLUSTER_NAME'_solr@' $_f
#sed -i -e 's@^SQL_CONNECTOR_JAR=.*@SQL_CONNECTOR_JAR=/usr/share/java/mysql-connector-java.jar@' $_f
# As "root" user
. /etc/hadoop/conf/hadoop-env.sh
/usr/hdp/`hdp-select versions | tail -1`/ranger-solr-plugin/enable-solr-plugin.sh
# 念のためタイムアウトを増加
sed -i 's/(sleep 5)/(sleep 30)/g' /opt/lucidworks-hdpsearch/solr/bin/solr
service solr restart
# Mpackの場合はAmbariからリスタート
# 確認
ls -l /opt/lucidworks-hdpsearch/solr/server/solr-webapp/webapp/WEB-INF/lib/*ranger*
lrwxrwxrwx 1 root root 93 Sep 6 09:18 /opt/lucidworks-hdpsearch/solr/server/solr-webapp/webapp/WEB-INF/lib/ranger-plugin-classloader-0.6.0.2.5.0.0-1245.jar -> /usr/hdp/2.5.0.0-1245/ranger-solr-plugin/lib/ranger-plugin-classloader-0.6.0.2.5.0.0-1245.jar
lrwxrwxrwx 1 root root 68 Sep 6 09:18 /opt/lucidworks-hdpsearch/solr/server/solr-webapp/webapp/WEB-INF/lib/ranger-solr-plugin-impl -> /usr/hdp/2.5.0.0-1245/ranger-solr-plugin/lib/ranger-solr-plugin-impl
lrwxrwxrwx 1 root root 91 Sep 6 09:18 /opt/lucidworks-hdpsearch/solr/server/solr-webapp/webapp/WEB-INF/lib/ranger-solr-plugin-shim-0.6.0.2.5.0.0-1245.jar -> /usr/hdp/2.5.0.0-1245/ranger-solr-plugin/lib/ranger-solr-plugin-shim-0.6.0.2.5.0.0-1245.jar
# AuthorizationにRangerを指定する（太字のところは変更する場合あり）
sudo -u solr /opt/lucidworks-hdpsearch/solr/server/scripts/cloud-scripts/zkcli.sh -zkhost ${_ZOOKEEPER_HOST}:2181 -cmd put /solr/security.json '{"authentication":{"class": "org.apache.solr.security.KerberosPlugin"},"authorization":{"class": "org.apache.ranger.authorization.solr.authorizer.RangerSolrAuthorizer"}}'
# AuthorizationにRangerを指定する（太字のところは変更する場合あり）
sudo -u solr /opt/lucidworks-hdpsearch/solr/server/scripts/cloud-scripts/zkcli.sh -zkhost ${_ZOOKEEPER_HOST}:2181 -cmd put /solr/security.json '{"authentication":{"class": "org.apache.solr.security.KerberosPlugin"},"authorization":{"class": "org.apache.ranger.authorization.solr.authorizer.RangerSolrAuthorizer"}}'
# Mpackの場合は、Ambariから再起動すると書き換えられてしまうので、下記のファイルを変更する必要あり
/var/lib/ambari-server/resources/mpacks/solr-ambari-mpack-*/common-services/SOLR/*/package/scripts/setup_solr_kerberos_auth.py
/var/lib/ambari-agent/cache/common-services/SOLR/*/package/scripts/setup_solr_kerberos_auth.py
# さらに、mpack 2.2.9からはAdvanced solr-securityから変更可能なはずだが、solr_security_jsonのところがsecurity_jsonになっている、かつ"Solr Security Json was found, it will not be overridden"
#sudo -u solr /opt/lucidworks-hdpsearch/solr/server/scripts/cloud-scripts/zkcli.sh -zkhost ${_ZOOKEEPER_HOST}:2181 -cmd clear /solr/security.json
# AmbariからRangerのプロパティを追加 custom ranger-admin-site
ranger.plugins.solr.serviceuser=solr
# core-siteにproxyuser
hadoop.proxyuser.solr.hosts=*
hadoop.proxyuser.solr.groups=*
# Mpackの場合は、Ambariから再起動すると書き換えられてしまうので、下記のファイルを変更する必要あり
/var/lib/ambari-agent/cache/common-services/SOLR/*/package/scripts/setup_solr_kerberos_auth.py
# さらに、mpack 2.2.9からはAdvanced solr-securityから変更可能なはずだが、solr_security_jsonのところがsecurity_jsonになっている、かつ"Solr Security Json was found, it will not be overridden"
#sudo -u solr /opt/lucidworks-hdpsearch/solr/server/scripts/cloud-scripts/zkcli.sh -zkhost ${_ZOOKEEPER_HOST}:2181 -cmd clear /solr/security.json
# AmbariからRangerのプロパティを追加 custom ranger-admin-site
ranger.plugins.solr.serviceuser=solr
# core-siteにproxyuser
hadoop.proxyuser.solr.hosts=*
hadoop.proxyuser.solr.groups=*
Ranger UIからSolrサービスを追加する
Service Name: ubu04c11_solr # clustername_service
Username: amb_ranger_admin # exists in x_portal_user
Password: r************d
Solr Url: http://$_SOLR_FQDN:8983/solr
#policy.download.auth.users = solr
#policy.download.auth.users = solr
Note:"/solr"が必要かどうかは不明。HWXとApacheで記述が異なる
Ambariを使わずにSolrを開始するには：
sudo -u solr /opt/lucidworks-hdpsearch/solr/bin/solr start -h node11.localdomain -cloud -z node13.localdomain:2181/solr -Dsolr.directoryFactory=HdfsDirectoryFactory -Dsolr.lock.type=hdfs -Dsolr.hdfs.home=hdfs://node12.localdomain:8020/solr -Dsolr.hdfs.confdir=/usr/hdp/current/hadoop-client/conf -Dsolr.hdfs.security.kerberos.enabled=true -Dsolr.hdfs.security.kerberos.keytabfile=/etc/security/keytabs/solr.service.keytab -Dsolr.hdfs.security.kerberos.principal=solr/node11.localdomain@EXAMPLE.COM -p 8983 -m 512m >> /var/log/service_solr/solr-service.log 2>&1# ステータスの確認
/opt/lucidworks-hdpsearch/solr/bin/solr status
Found 1 Solr nodes:
Solr process 2216 running on port 8983
INFO - 2017-09-06 09:37:14.907; org.apache.solr.util.SolrCLI; Set HttpClientConfigurer from: org.apache.solr.client.solrj.impl.Krb5HttpClientConfigurer
INFO - 2017-09-06 09:37:15.116; org.apache.solr.client.solrj.impl.Krb5HttpClientConfigurer; Setting up SPNego auth with config: /etc/solr/conf/solr_server_jaas.conf
{
"solr_home":"/etc/solr/data_dir",
"version":"5.5.2 8e5d40b22a3968df065dfc078ef81cbb031f0e4a - sarowe - 2016-06-21 11:44:11",
"startTime":"2017-09-06T09:36:49.46Z",
"uptime":"0 days, 0 hours, 0 minutes, 26 seconds",
"memory":"84.6 MB (%17.3) of 490.7 MB",
"cloud":{
"ZooKeeper":"node13.localdomain:2181/solr",
"liveNodes":"1",
"collections":"1"}}
# Ranger Solr pluginのPolicyがダウンロドできるか確認
su - solr
kinit -kt /etc/security/keytabs/solr.service.keytab solr/sandbox.hortonworks.com
curl -v -u amb_ranger_admin "http://sandbox.hortonworks.com:6080/service/plugins/secure/policies/download/Sandbox_solr"
curl -v --negotiate -u: "http://sandbox.hortonworks.com:6080/service/plugins/secure/policies/download/Sandbox_solr"
よくあるエラー
xasecure.policymgr.clientssl.truststoreがPluginのサービスユーザから読めない
https://issues.apache.org/jira/browse/SOLR-8335
hdfs dfs -ls -R /solr/ | grep write.lock
-rw-r--r-- 3 solr hdfs 0 2017-08-31 08:26 hdfs://node12.localdomain:8020/solr/collection1/core_node1/data/index/write.lock
-rw-r--r-- 3 solr hdfs 0 2017-08-31 08:26 hdfs://node12.localdomain:8020/solr/collection1/core_node2/data/index/write.lock
2. /opt/lucidworks-hdpsearch/solr/server/scripts/cloud-scripts/zkcli.sh -zkhost node13.localdomain:2181 -cmd makepath /solr
Exception in thread "main" org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /solr
at org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
at org.apache.solr.common.cloud.SolrZkClient$10.execute(SolrZkClient.java:501)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:498)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:455)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:442)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:398)
at org.apache.solr.cloud.ZkCLI.main(ZkCLI.java:258)
3. service solr start
/opt/solr not found! Please check the SOLR_INSTALL_DIR setting in your /etc/init.d/solr script.
/etc/default/solr.in.sh not found! Please check the SOLR_ENV setting in your /etc/init.d/solr script.
ln -s /opt/lucidworks-hdpsearch/solr /opt/solr
ln -s /opt/lucidworks-hdpsearch/solr/bin/solr.in.sh /etc/default/solr.in.sh
4. org.apache.ranger.authorization.hadoop.utils.RangerCredentialProvider (RangerCredentialProvider.java:72) - Unable to get the Credential Provider from the Configuration
java.lang.IllegalArgumentException: The value of property hadoop.security.credential.provider.path must not be null
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:92)
at org.apache.hadoop.conf.Configuration.set(Configuration.java:1010)
at org.apache.hadoop.conf.Configuration.set(Configuration.java:991)
at org.apache.ranger.authorization.hadoop.utils.RangerCredentialProvider.getCredentialProviders(RangerCredentialProvider.java:68)
...
/usr/hdp/2.6.0.3-8/ranger-solr-plugin/install/conf.templates/enable/ranger-policymgr-ssl.xml
/opt/lucidworks-hdpsearch/solr/server/solr-webapp/webapp/WEB-INF/classes/ranger-policymgr-ssl.xml
ln -s /opt/lucidworks-hdpsearch/solr/server/solr-webapp/webapp/WEB-INF/classes/ranger-policymgr-ssl.xml /etc/solr/conf/ranger-policymgr-ssl.xml
5. SLF4J: Failed toString() invocation on an object of type [org.apache.solr.servlet.HttpSolrCall$2]
java.lang.NullPointerException
at org.apache.solr.servlet.HttpSolrCall$2.toString(HttpSolrCall.java:1001)
at org.slf4j.helpers.MessageFormatter.safeObjectAppend(MessageFormatter.java:305)
at org.slf4j.helpers.MessageFormatter.deeplyAppendParameter(MessageFormatter.java:277)
at org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:231)
at org.slf4j.helpers.MessageFormatter.format(MessageFormatter.java:152)
at org.slf4j.impl.Log4jLoggerAdapter.info(Log4jLoggerAdapter.java:345)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:431)
...
https://issues.apache.org/jira/browse/SOLR-10630
https://issues.apache.org/jira/browse/RANGER-1446
プロセス例：
solr 6218 8.5 3.3 5852532 547840 ? Sl 10:04 0:20 /usr/jdk64/jdk1.8.0_77/bin/java -server -Xms512m -Xmx512m -XX:NewRatio=3 -XX:SurvivorRatio=4 -XX:TargetSurvivorRatio=90 -XX:MaxTenuringThreshold=8 -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:ConcGCThreads=4 -XX:ParallelGCThreads=4 -XX:+CMSScavengeBeforeRemark -XX:PretenureSizeThreshold=64m -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=50 -XX:CMSMaxAbortablePrecleanTime=6000 -XX:+CMSParallelRemarkEnabled -XX:+ParallelRefProcEnabled -XX:MaxDirectMemorySize=20g -XX:+UseLargePages -verbose:gc -XX:+PrintHeapAtGC -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -Xloggc:/var/log/solr/solr_gc.log -DzkClientTimeout=15000 -DzkHost=node13.localdomain:2181/solr -Djetty.port=8983 -DSTOP.PORT=7983 -DSTOP.KEY=solrrocks -Dhost=node11.localdomain -Duser.timezone=UTC -Djetty.home=/opt/lucidworks-hdpsearch/solr/server -Dsolr.solr.home=/etc/solr/data_dir -Dsolr.install.dir=/opt/lucidworks-hdpsearch/solr -Dlog4j.configuration=file:/etc/solr/conf/log4j.properties -Xss256k -Dsolr.directoryFactory=HdfsDirectoryFactory -Dsolr.lock.type=hdfs -Dsolr.hdfs.home=hdfs://node12.localdomain:8020/solr -Dsolr.hdfs.confdir=/usr/hdp/current/hadoop-client/conf -Dsolr.hdfs.security.kerberos.enabled=true -Dsolr.hdfs.security.kerberos.keytabfile=/etc/security/keytabs/solr.service.keytab -Dsolr.hdfs.security.kerberos.principal=solr/node11.localdomain@EXAMPLE.COM -Dsolr.authentication.httpclient.configurer=org.apache.solr.client.solrj.impl.Krb5HttpClientConfigurer -Djava.security.auth.login.config=/etc/solr/conf/solr_server_jaas.conf -Dsolr.kerberos.cookie.domain=node11.localdomain -Dsolr.kerberos.cookie.portaware=true -Dsolr.kerberos.principal=HTTP/node11.localdomain@EXAMPLE.COM -Dsolr.kerberos.keytab=/etc/security/keytabs/spnego.service.keytab -XX:OnOutOfMemoryError=/opt/lucidworks-hdpsearch/solr/bin/oom_solr.sh 8983 /var/log/solr -jar start.jar --module=httpAmbari-infraと共存するには
grep 'ps aux' /opt/lucidworks-hdpsearch/solr/bin/solr | grep 'start'" | grep -v ambari-infra"を追加する
auth_to_localを設定するには
SOLR_AUTHENTICATION_OPTS=" -DauthenticationPlugin=org.apache.solr.security.KerberosPlugin -Djava.security.auth.login.config=${SOLR_JAAS_FILE} -Dsolr.kerberos.principal=${SOLR_KERB_PRINCIPAL} -Dsolr.kerberos.keytab=${SOLR_KERB_KEYTAB} -Dsolr.kerberos.cookie.domain=${SOLR_HOST} -Dhost=${SOLR_HOST} -Dsolr.kerberos.name.rules=RULE:[1:\$1@\$0](.*EXAMPLE.COM)s/@.*//LDEFAULT"mpack問題
mpack 2.2.8だと、RangerがSSLを使用しているとうまくいかない模様そもそもmpackインストール後、security.jsonが変更できない（Ambariのデフォルトに書き換えられる）
Pluginのインストール先：/usr/hdp/2.6.0.3-8/ranger-solr-plugin
不親切にもinstall.propertiesを自分で設定する必要あり
https://issues.apache.org/jira/browse/RANGER-1446
https://issues.apache.org/jira/browse/RANGER-1658
HDP 2.6.3で修正済みの模様
Sandboxにmpackをインストールしようとすると、
ambari-server install-mpack --mpack=/tmp/solr-service-mpack-2.2.9.tar.gz --verbose
...
INFO: Loading properties from /etc/ambari-server/conf/ambari.properties
source:/var/lib/ambari-server/resources/mpacks/solr-ambari-mpack-2.2.9/common-services/SOLR/5.5.4
link_name:/var/lib/ambari-server/resources/common-services/SOLR/5.5.4
Traceback (most recent call last):
File "/usr/sbin/ambari-server.py", line 941, in <module>
... (snip)...
sudo.symlink(src_path, dest_link)
File "/usr/lib/python2.6/site-packages/resource_management/core/sudo.py", line 125, in symlink
os.symlink(source, link_name)
OSError: [Errno 17] File exists
または、
ERROR: Management pack solr-ambari-mpack-2.2.N already installed!
ERROR: Exiting with exit code -1.
REASON: Management pack solr-ambari-mpack-2.2.N already installed!
find /var/lib/ambari-server -type l -ls | grep SOLR
#rm -rf /var/lib/ambari-server/resources/mpacks/solr-ambari-mpack*
#rm /var/lib/ambari-server/resources/common-services/SOLR/5.5.2.2.*
# 多分これだけで大丈夫
rm /var/lib/ambari-server/resources/stacks/HDP/2.4/services/SOLR
#rm -rf /var/lib/ambari-server/resources/extensions/SOLR/*
NOTE: ambari-server backup でバックアップが取れる！
/etc/solr/conf/solr_server_jaas.conf
log4j.logger.org.apache.ranger.authorization.solr.authorizer.RangerSolrAuthorizer=WARN
0 件のコメント:
コメントを投稿