[pgpool-hackers: 3544] Re: [PATCH] Feature: Support for CRL (Certificate Revocation List)
Tatsuo Ishii
ishii at sraoss.co.jp
Wed Mar 11 07:04:20 JST 2020
Unfortunately the regression test failed again.
$ ./regress.sh 024
creating pgpool-II temporary installation ...
moving pgpool_setup to temporary installation path ...
moving watchdog_setup to temporary installation path ...
using pgpool-II at /home/t-ishii/work/Pgpool-II/current/pgpool2/src/test/regression/temp/installed
*************************
REGRESSION MODE : install
PGPOOL-II : /home/t-ishii/work/Pgpool-II/current/pgpool2/src/test/regression/temp/installed
PostgreSQL bin : /usr/local/pgsql/bin
PostgreSQL Major version : 12
pgbench : /usr/local/pgsql/bin/pgbench
PostgreSQL jdbc : /usr/local/pgsql/share/postgresql-9.2-1003.jdbc4.jar
*************************
testing 024.cert_auth...failed.
out of 1 ok:0 failed:1 timeout:0
regression log attached.
> Thanks Usama for feedback. updated patch with suggested changed attached.
> Please see comments inline.
>
> Ishii,
> IMO, tests were failing because some certificates generation and signing
> was using global configuration. Previously, I only provided
> minimum configuration for CRL generation.
> Now I provided minimum custom configuration for Cert generation and Signing
> too. I hope now that last tests should be passing now.
>
> Regards
> Umar Hayat
>
>
> On Wed, Mar 4, 2020 at 7:19 PM Muhammad Usama <m.usama at gmail.com> wrote:
>
>> Hi
>>
>> I have looked into the patch and here are my two cents.
>>
>> 1- Does the "Certificate Revocation List (CRL)" have any effect if we do
>> not configure CA certificates?
>> Meaning what would be the behavior of the system if someone specifies
>> "ssl_crl_file" but leave out the ssl_ca_cert config?
>>
>> If CA is not configure then certification validation will fail while
> validating certification.
>
>>
>> 2- In the patch you are loading the Certificate Revocation List after
>> configuring
>> SSL library to ask for client certificates
>> i.e after calling SSL_CTX_set_verify and SSL_CTX_set_client_CA_list
>> functions
>>
>> Although I am not able to find any related SSL documentation on the
>> subject and not sure if it can cause any problem. But in the normal
>> sequence, we always load
>> the CA and other certificated before SSL_CTX_set_verify and
>> SSL_CTX_set_client_CA_list functions.
>>
>> So I guess the safe bet is to move the loading of Certificate Revocation
>> List before
>> SSL_CTX_set_verify.
>>
>> I wasn't sure either that would matter as we are just registering event at
> that state, but as suggest i reorder the code as you suggested ( as
> Postgres does too )
>
>> 3- The comment /* OpenSSL 0.96 does not support X509_V_FLAG_CRL_CHECK */
>> mentions
>> OpenSSL version as 0.96, which is wrong and should be 0.9.6
>>
>> Comment fixed.
>
>> 4- In the test case you are only exporting PGSSLCERT and PGSSLKEY
>> environment variables
>> that means the test case will try to pick up the root certificate from the
>> default location
>> and on the systems where the root certificate would not be found there,
>> the test case will fail.
>>
>> So I think you need to modify the test case and use PGSSLROOTCERT
>> environment variable to
>> explicitly specify the root certificate as well.
>>
>> PGSSLROOTCERT defined.
>
>> These comments are on top of Ishii-San's review comments.
>>
>> Thanks
>> Best Regards
>> Muhammad Usama
>>
>>
>> On Wed, Mar 4, 2020 at 12:11 PM Tatsuo Ishii <ishii at sraoss.co.jp> wrote:
>>
>>> Hi Umar,
>>>
>>> Any update on this?
>>>
>>> From: Tatsuo Ishii <ishii at sraoss.co.jp>
>>> Subject: [pgpool-hackers: 3523] Re: [PATCH] Feature: Support for CRL
>>> (Certificate Revocation List)
>>> Date: Fri, 28 Feb 2020 14:27:55 +0900 (JST)
>>> Message-ID: <20200228.142755.205379439290164558.t-ishii at sraoss.co.jp>
>>>
>>> > Here are comments on your patch.
>>> >
>>> > - There are some extra trailing spaces.
>>> >
>>> > $ git apply ~/crl_support_with_testcase.diff
>>> > /home/t-ishii/crl_support_with_testcase.diff:42: trailing whitespace.
>>> > Specifies the name of the file containing the SSL server
>>> > /home/t-ishii/crl_support_with_testcase.diff:43: trailing whitespace.
>>> > certificate revocation list (CRL). The default is empty,
>>> > /home/t-ishii/crl_support_with_testcase.diff:199: new blank line at EOF.
>>> > +
>>> > warning: 3 lines add whitespace errors.
>>> >
>>> > - The pached source code compililes without any error.
>>> >
>>> > - the regression test (024.cert_auth) failed.
>>> >
>>> > ./regress.sh 024
>>> > creating pgpool-II temporary installation ...
>>> > moving pgpool_setup to temporary installation path ...
>>> > moving watchdog_setup to temporary installation path ...
>>> > using pgpool-II at
>>> /home/t-ishii/work/Pgpool-II/current/pgpool2/src/test/regression/temp/installed
>>> > *************************
>>> > REGRESSION MODE : install
>>> > PGPOOL-II :
>>> /home/t-ishii/work/Pgpool-II/current/pgpool2/src/test/regression/temp/installed
>>> > PostgreSQL bin : /usr/local/pgsql/bin
>>> > PostgreSQL Major version : 12
>>> > pgbench : /usr/local/pgsql/bin/pgbench
>>> > PostgreSQL jdbc :
>>> /usr/local/pgsql/share/postgresql-9.2-1003.jdbc4.jar
>>> > *************************
>>> > testing 024.cert_auth...failed.
>>> > out of 1 ok:0 failed:1 timeout:0
>>> >
>>> > This is Ubuntu 18.04.4 LTS.
>>> >
>>> > $ openssl version
>>> > OpenSSL 1.1.1 11 Sep 2018
>>> >
>>> > Please find attached log file for the
>>> > regression test.
>>> >
>>> > Best regards,
>>> > --
>>> > Tatsuo Ishii
>>> > SRA OSS, Inc. Japan
>>> > English: http://www.sraoss.co.jp/index_en.php
>>> > Japanese:http://www.sraoss.co.jp
>>> >
>>> >> Hi Umar,
>>> >>
>>> >> I seemed to miss your last email. I will take care your patch
>>> >> tomorrow morning.
>>> >>
>>> >> Best regards,
>>> >> --
>>> >> Tatsuo Ishii
>>> >> SRA OSS, Inc. Japan
>>> >> English: http://www.sraoss.co.jp/index_en.php
>>> >> Japanese:http://www.sraoss.co.jp
>>> >>
>>> >>> Hi Tatsuo,
>>> >>> Any update for last patch?
>>> >>> I will be sending more patches in the same area of SSL ( for few other
>>> >>> features ) and the those patches might create conflict on merge.
>>> >>>
>>> >>> Regards,
>>> >>> Umar Hayat
>>> >>> Principal Software Engineer
>>> >>> EnterpriseDB: https://www.enterprisedb.com
>>> >>>
>>> >>> On Wed, Feb 19, 2020 at 1:39 PM Umar Hayat <m.umarkiani at gmail.com>
>>> wrote:
>>> >>>
>>> >>>> Hi Tatsuo,
>>> >>>> Please find the attached updated patch with following changes:
>>> >>>> 1. Updated the description of '*ssl_crl_file'* configuration
>>> variable.
>>> >>>> 2. Updated test case '024.cert_auth' which verify valid CRL and
>>> invalid
>>> >>>> CRL ( CRL with revocation entry )
>>> >>>>
>>> >>>> Regards,
>>> >>>> Umar Hayat
>>> >>>>
>>> >>>>
>>> >>>> On Thu, Feb 13, 2020 at 3:43 AM Tatsuo Ishii <ishii at sraoss.co.jp>
>>> wrote:
>>> >>>>
>>> >>>>> > I just followed the description pattern used for other ssl
>>> variables. We
>>> >>>>> > can use PostgreSQL doc if we remove following two line from that:
>>> >>>>> > "Relative paths are relative to the data
>>> >>>>> > directory. This parameter can only be set in the postgresql.conf
>>> file
>>> >>>>> > or on the server command line.
>>> >>>>> > "
>>> >>>>>
>>> >>>>> Sounds good to me.
>>> >>>>>
>>> >>>>> > - It would be nice to include regression test patch. See
>>> >>>>> >> src/test/023.ssl_connection for an example.
>>> >>>>> >>
>>> >>>>> >
>>> >>>>> > Sure, I will create and send test patch in
>>> src/test/023.ssl_connection.
>>> >>>>> > I will try to generate CRL file for existing certification file
>>> in this
>>> >>>>> > this test. If thats not possible, then I have to generate new
>>> >>>>> certification
>>> >>>>> > and CRL file.
>>> >>>>>
>>> >>>>> Thank you. Looking forward to the new patch.
>>> >>>>>
>>> >>>>> Best regards,
>>> >>>>> --
>>> >>>>> Tatsuo Ishii
>>> >>>>> SRA OSS, Inc. Japan
>>> >>>>> English: http://www.sraoss.co.jp/index_en.php
>>> >>>>> Japanese:http://www.sraoss.co.jp
>>> >>>>>
>>> >>>>
>>> >> _______________________________________________
>>> >> pgpool-hackers mailing list
>>> >> pgpool-hackers at pgpool.net
>>> >> http://www.pgpool.net/mailman/listinfo/pgpool-hackers
>>> _______________________________________________
>>> pgpool-hackers mailing list
>>> pgpool-hackers at pgpool.net
>>> http://www.pgpool.net/mailman/listinfo/pgpool-hackers
>>>
>>
-------------- next part --------------
./cert.sh: ? 13: ca: ????????????
Generating a RSA private key
...............+++++
...............................................+++++
writing new private key to 'root.key'
-----
Generating a RSA private key
..+++++
..................................................................................................................................................+++++
writing new private key to 'server.key'
-----
Using configuration from crl_openssl.conf
Check that the request matches the signature
Signature ok
Certificate Details:
Serial Number: 4096 (0x1000)
Validity
Not Before: Mar 10 21:58:17 2020 GMT
Not After : Mar 20 21:58:17 2021 GMT
Subject:
commonName = postgresql
Certificate is to be certified until Mar 20 21:58:17 2021 GMT (375 days)
Write out database with 1 new entries
Data Base Updated
Generating a RSA private key
..............+++++
........................................+++++
writing new private key to 'frontend.key'
-----
Using configuration from crl_openssl.conf
Check that the request matches the signature
Signature ok
Certificate Details:
Serial Number: 4097 (0x1001)
Validity
Not Before: Mar 10 21:58:17 2020 GMT
Not After : Mar 20 21:58:17 2021 GMT
Subject:
commonName = t-ishii
Certificate is to be certified until Mar 20 21:58:17 2021 GMT (375 days)
Write out database with 1 new entries
Data Base Updated
Using configuration from crl_openssl.conf
Using configuration from crl_openssl.conf
Revoking Certificate 1001.
Data Base Updated
Using configuration from crl_openssl.conf
creating test environment...PostgreSQL major version: 120
Starting set up in streaming replication mode
creating startall and shutdownall
creating failover script
creating database cluster /home/t-ishii/work/Pgpool-II/current/pgpool2/src/test/regression/tests/024.cert_auth/testdir/data0...done.
update postgresql.conf
creating pgpool_remote_start
creating basebackup.sh
creating recovery.conf
temporarily start data0 cluster to create extensions
temporarily start pgpool-II to create standby nodes
node_id | hostname | port | status | lb_weight | role | select_cnt | load_balance_node | replication_delay | replication_state | replication_sync_state | last_status_change
---------+----------+-------+--------+-----------+---------+------------+-------------------+-------------------+-------------------+------------------------+---------------------
0 | /tmp | 11002 | up | 1.000000 | primary | 0 | true | 0 | | | 2020-03-11 06:58:19
(1 row)
creating follow master script
node_id | hostname | port | status | lb_weight | role | select_cnt | load_balance_node | replication_delay | replication_state | replication_sync_state | last_status_change
---------+----------+-------+--------+-----------+---------+------------+-------------------+-------------------+-------------------+------------------------+---------------------
0 | /tmp | 11002 | up | 1.000000 | primary | 0 | true | 0 | | | 2020-03-11 06:58:19
(1 row)
shutdown all
pgpool-II setting for streaming replication mode is done.
To start the whole system, use /home/t-ishii/work/Pgpool-II/current/pgpool2/src/test/regression/tests/024.cert_auth/testdir/startall.
To shutdown the whole system, use /home/t-ishii/work/Pgpool-II/current/pgpool2/src/test/regression/tests/024.cert_auth/testdir/shutdownall.
pcp command user name is "t-ishii", password is "t-ishii".
Each PostgreSQL, pgpool-II and pcp port is as follows:
#1 port is 11002
pgpool port is 11000
pcp port is 11001
The info above is in README.port.
done.
waiting for server to start....14086 2020-03-11 06:58:20 JST LOG: starting PostgreSQL 12.0 on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0, 64-bit
14086 2020-03-11 06:58:20 JST LOG: listening on IPv4 address "0.0.0.0", port 11002
14086 2020-03-11 06:58:20 JST LOG: listening on IPv6 address "::", port 11002
14086 2020-03-11 06:58:20 JST LOG: listening on Unix socket "/tmp/.s.PGSQL.11002"
14086 2020-03-11 06:58:20 JST LOG: redirecting log output to logging collector process
14086 2020-03-11 06:58:20 JST HINT: Future log output will appear in directory "log".
done
server started
?column?
----------
1
(1 row)
2020-03-11 06:58:21: pid 14132: LOG: SSL certificate authentication for user "t-ishii" with Pgpool-II is successful
Checking cert auth between Pgpool-II and frontend was ok.
2020-03-11 06:58:21: pid 14149: LOG: stop request sent to pgpool. waiting for termination...
.done.
waiting for server to shut down.... done
server stopped
waiting for server to start....14157 2020-03-11 06:58:23 JST LOG: starting PostgreSQL 12.0 on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0, 64-bit
14157 2020-03-11 06:58:23 JST LOG: listening on IPv4 address "0.0.0.0", port 11002
14157 2020-03-11 06:58:23 JST LOG: listening on IPv6 address "::", port 11002
14157 2020-03-11 06:58:23 JST LOG: listening on Unix socket "/tmp/.s.PGSQL.11002"
14157 2020-03-11 06:58:23 JST LOG: redirecting log output to logging collector process
14157 2020-03-11 06:58:23 JST HINT: Future log output will appear in directory "log".
done
server started
?column?
----------
1
(1 row)
2020-03-11 06:58:24: pid 14203: LOG: SSL certificate authentication for user "t-ishii" with Pgpool-II is successful
Checking cert auth between Pgpool-II and frontend with clean CRL was ok.
2020-03-11 06:58:24: pid 14220: LOG: stop request sent to pgpool. waiting for termination...
.done.
waiting for server to shut down.... done
server stopped
waiting for server to start....14227 2020-03-11 06:58:25 JST LOG: starting PostgreSQL 12.0 on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0, 64-bit
14227 2020-03-11 06:58:25 JST LOG: listening on IPv4 address "0.0.0.0", port 11002
14227 2020-03-11 06:58:25 JST LOG: listening on IPv6 address "::", port 11002
14227 2020-03-11 06:58:25 JST LOG: listening on Unix socket "/tmp/.s.PGSQL.11002"
14227 2020-03-11 06:58:25 JST LOG: redirecting log output to logging collector process
14227 2020-03-11 06:58:25 JST HINT: Future log output will appear in directory "log".
done
server started
?column?
----------
1
(1 row)
Checking cert auth between Pgpool-II and frontend with revoked entry in CRL failed.
2020-03-11 06:58:26: pid 14289: LOG: stop request sent to pgpool. waiting for termination...
.done.
waiting for server to shut down.... done
server stopped
More information about the pgpool-hackers
mailing list