Hadoop and DevSecOps notes & tips: 10月 2017

2017年10月27日金曜日

MacからWindows ADユーザをldapsearch + Kerberos(GSSAPI)で検索してみる

krb5.confを編集

[realms]にADのRealmとサーバアドレスを追加（ホスト名の方がベターかも）
$ sudo vim /private/etc/krb5.conf
HDP.LOCALDOMAIN = {
admin_server = 192.168.0.21
kdc = 192.168.0.21
}

ログインしてチケットを確認

$ kinit
hosako@HDP.LOCALDOMAIN's password:
HW11970:~ hosako$ klist
Credentials cache: API:301A5EDD-3897-4E1E-A4CE-52C35E56D494
Principal: hosako@HDP.LOCALDOMAIN

Issued Expires Principal
Oct 27 08:09:07 2017 Oct 27 18:09:07 2017 krbtgt/HDP.LOCALDOMAIN@HDP.LOCALDOMAIN

自分を検索してみる

IPだとうまく行きません
$ ldapsearch -Y GSSAPI -R HDP.LOCALDOMAIN -U "hosako@hdp.localdomain" -b "dc=hdp,dc=localdomain" -h 192.168.0.21 "(sAMAccountName=hosako)"
SASL/GSSAPI authentication started
ldap_sasl_interactive_bind_s: Local error (-2)
additional info: SASL(-1): generic failure: GSSAPI Error: Miscellaneous failure (see text (Server (krbtgt/168.0.21@HDP.LOCALDOMAIN) unknown while looking up ...

FQDNだとうまく行きます。
$ ldapsearch -LL -Y GSSAPI -b "dc=hdp,dc=localdomain" -h WIN-TEST.hdp.localdomain "(sAMAccountName=hosako)" dn
SASL/GSSAPI authentication started
SASL username: hosako@HDP.LOCALDOMAIN
SASL SSF: 112
SASL data security layer installed.
version: 1

dn: CN=Hajime Osako,CN=Users,DC=hdp,DC=localdomain
...

Service Principal Nameでも検索してみる
$ ldapsearch -LL -Y GSSAPI -b "dc=hdp,dc=localdomain" -h WIN-TEST.hdp.localdomain "(serviceprincipalname=HTTP*)" dn
SASL/GSSAPI authentication started
SASL username: hosako@HDP.LOCALDOMAIN
SASL SSF: 112
SASL data security layer installed.
version: 1

dn: CN=HTTP/sandbox.hortonworks.com,OU=Hadoop,DC=hdp,DC=localdomain
...

2017年10月20日金曜日

検証中：HDPとADのテスト環境を作る(LDAPS/Forest)

1. Create 1 Linux VM (Sandbox), 2 Windows VMs

Use private IPs (10.1.0.x) to connect each other (means need to edit hosts file)

2. Set up AD as a new Forest on one Windows (HDP.LOCALDOMAIN)

Add "Active Directory Domain Services" (shouldn't install AD CS at same time)
Then when configure, select Add a new forest

3. Set up ldaps (AD CS Configuration)

Ref: http://gregtechnobabble.blogspot.in/2012/11/enabling-ldap-ssl-in-windows-2012-part-1.html

Open Server Manager
Add roles and features
Role-based or feature-based installation
Tick "Active Directory Certificate Services" and required services
Nothing to add in "Select features" page, so "Next" twice
In Select role services, select "Certification Authority"
In Confirm installation selections, tick "Restart the destination server ..."
Install! and close
In Server Manager, in the top right, should see the flag with a warning icon
Click this, and select "Post-deployment Configuration", or click "AD CS"
Click on "Configure Active Directory Certificate Services..."
Check the credential which will be used to configure, then Next
Tick "Certification Authority"
Somehow can't click "Enterprise CA"
If you are not able to select “Enterprise CA”, add “Enterprise Admins” for your user. Then you will be able to select.
Root CA, and Next
Create private key, Next
Default (sha1), Next
In "CA Name" page, review, Next.... until Confirmation page.
Click Configure
This shouldn't take long (a few seconds), then Close.
Reboot

Export root CA certificate (to use in truststore/browser etc.)

Start certsrv (Certification Authority console)
Right click the server, then Properties
From General tab, click "View Certificate" button
From Details tab, click "Copy to File" button

確認:

ldapsearch -h 192.168.0.21 -D Hortonworks@hdp.localdomain -W -b 'DC=HDP,DC=LOCALDOMAIN' '(sAMAccountName=Hortonworks*)'
ldapsearch -H ldaps://192.168.0.21:636 -D Hortonworks@hdp.localdomain -W -b 'DC=HDP,DC=LOCALDOMAIN' '(sAMAccountName=Hortonworks*)'

If you get certification error because of self-cert, AD is serving on LDAPS port.

4. Create a new container "hadoop"

ref: https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Enable%20Kerberos%20in%20Ambari%20with%20Existing%20Active%20Directory

参考:

https://gist.github.com/magnetikonline/0ccdabfec58eb1929c997d22e7341e45
https://rms-digicert.ne.jp/howto/install/install_directory-ldap-2012.html

証明書の出力方法
openssl s_client -showcerts -connect 192.168.8.21:636

2017年10月12日木曜日

HDP Sandbox 2.6.0でKnox LDAP DemoをRanger Usersyncに使ってみる

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.2/bk_security/content/ranger_user_sync_settings.html

まず、Demo LDAPが起動しているか確認

[root@sandbox ~]# netstat -lopen | grep 33389
tcp 0 0 0.0.0.0:33389 0.0.0.0:* LISTEN 522 1325846 84375/java off (0.00/0/0)

[root@sandbox ~]# ldapsearch -x -h `hostname -f`:33389 -D 'uid=admin,ou=people,dc=hadoop,dc=apache,dc=org' -w admin-password -s sub '(uid=admin)'
# extended LDIF
#
# LDAPv3
# base <> (default) with scope subtree
# filter: (uid=admin)
# requesting: ALL
#

# admin, people, hadoop.apache.org
dn: uid=admin,ou=people,dc=hadoop,dc=apache,dc=org
sn: Admin
cn: Admin
objectclass: top
objectclass: person
objectclass: organizationalPerson
objectclass: inetOrgPerson
userpassword:: YWRtaW4tcGFzc3dvcmQ=
uid: admin

# search result
search: 2
result: 0 Success

# numResponses: 2
# numEntries: 1

Ambari UIで、UserSyncを設定する

http://sandbox.hortonworks.com:8080/api/v1/clusters/Sandbox/configurations/service_config_versions?service_name=RANGER&is_current=true

...

"type" : "ranger-ugsync-site",
...
"properties" : {
    "ranger.usersync.credstore.filename" : "/usr/hdp/current/ranger-usersync/conf/ugsync.jceks",
    "ranger.usersync.enabled" : "true",
    ...
    "ranger.usersync.group.searchenabled" : "false",

...

    "ranger.usersync.group.usermapsyncenabled" : "false",
    ...
    "ranger.usersync.ldap.bindalias" : "testldapalias",
    "ranger.usersync.ldap.binddn" : "uid=admin,ou=people,dc=hadoop,dc=apache,dc=org",
    "ranger.usersync.ldap.bindkeystore" : "",
    "ranger.usersync.ldap.deltasync" : "true",
    "ranger.usersync.ldap.groupname.caseconversion" : "none",
    "ranger.usersync.ldap.ldapbindpassword" : "SECRET:ranger-ugsync-site:4:ranger.usersync.ldap.ldapbindpassword",
    "ranger.usersync.ldap.referral" : "ignore",
    "ranger.usersync.ldap.searchBase" : "dc=hadoop,dc=apache,dc=org",
    "ranger.usersync.ldap.url" : "ldap://sandbox.hortonworks.com:33389",
    "ranger.usersync.ldap.user.groupnameattribute" : "memberof, ismemberof",
    "ranger.usersync.ldap.user.nameattribute" : "uid",
    "ranger.usersync.ldap.user.objectclass" : "person",
    "ranger.usersync.ldap.user.searchbase" : "dc=hadoop,dc=apache,dc=org",
    "ranger.usersync.ldap.user.searchfilter" : "(objectclass=person)",
    "ranger.usersync.ldap.user.searchscope" : "sub",
    "ranger.usersync.ldap.username.caseconversion" : "none",
    ...
    "ranger.usersync.user.searchenabled" : "false"
},

確認

sudo -u ranger java -cp "/usr/hdp/current/ranger-admin/cred/lib/*" org.apache.ranger.credentialapi.buildks list -provider /usr/hdp/current/ranger-usersync/conf/ugsync.jceks

2017年10月9日月曜日

HDPのKnox LDAP Demoを単独で実行する

mkdir conf
cp /etc/knox/conf/users.ldif ./conf/
cp /usr/hdp/current/knox-server/bin/ldap.jar ./
cp /usr/hdp/current/knox-server/lib/gateway-demo-ldap-*.jar ./
cp /usr/hdp/current/knox-server/dep/apacheds-all-*.jar ./

# わざと失敗して、ldap.cfgファイルを作成する
java -jar ./ldap.jar

grep class.path ldap.cfg
class.path=./*.jar;../lib/*.jar;../dep/*.jar;../ext;../ext/*.jar

ls -l
total 10356
-rw-r--r-- 1 root root 10527192 Oct 9 01:20 apacheds-all-2.0.0-M16.jar
drwxr-xr-x 2 root root 4096 Oct 9 01:11 conf
-rw-r--r-- 1 root root 37589 Oct 9 01:18 gateway-demo-ldap-0.12.0.2.6.0.3-8.jar
-rw-r--r-- 1 root root 321 Oct 9 01:20 ldap.cfg
-rw-r--r-- 1 root root 23750 Oct 9 01:11 ldap.jar

nohup java -jar ./ldap.jar &

Ref:

https://repo.hortonworks.com/service/local/repositories/central/content/org/apache/knox/gateway-demo-ldap-launcher/1.4.0/gateway-demo-ldap-launcher-1.4.0.jar

https://repo.hortonworks.com/service/local/repositories/central/content/org/apache/knox/gateway-demo-ldap/1.4.0/gateway-demo-ldap-1.4.0.jar