#7 Postgres-XC crashes with failed to find proc 0x7fb0cb34f7c0 in ProcArray and sorry, too many clients already

Diancheng Wang — Wed, 15 Apr 2015 06:44:45 -0000

attachment patch should fix this bug.

#7 Postgres-XC crashes with failed to find proc 0x7fb0cb34f7c0 in ProcArray and sorry, too many clients already

yazun — Wed, 05 Nov 2014 16:45:15 -0000

Dear Koichi,

I was hit by this bug too. Do you plan to apply the patch anytime soon?

According to wikipedia, the licensing should not be an issue here as XL uses MPL version 2.0:

http://en.wikipedia.org/wiki/Mozilla_Public_License

"In the absence of patents, MPL-licensed code can be freely used, altered, and redistributed. Versions with patented code can still be used, transferred, and even sold, but cannot be altered without special permission."

happily, it's not patented and I would take word 'use' as permission to apply it as a patch elsewhere as it is constrained to the 'component'.

But as usual with licensing it is confusing. Is there any chance to get a clear statement from XC and XL teams whether patches to already existing code in XC are moot if it comes to re-applying from MPL to BSD or there is chance for a mutual agreement to speed up development of both projects?

Cheers,
Chris.

Postgres-XC crashes with failed to find proc 0x7fb0cb34f7c0 in ProcArray and sorry, too many clients already

Mladen Stefanov — Tue, 21 Oct 2014 11:05:52 -0000

Hello,

I am really devastated. For the last week, week and half, I was trying to ran successfully almost all postgres-xc versions 1.1, 1.1.1, 1.2, 1.2.1...etc and the only one which runs without any problems was 1.0.4.
My setup is based on 4 AWS instances with Ubuntu 14.04 LTS on them.
2xData-Nodes 1xCordinator 1xGTM server. While I was strugling with it I tried many approaches like DL the code and compile it, install ubuntu package and so on.

Here is my problem and steps I do.
1. DL the code from sourceforge.net
2. unpak and do ./configure, make and make install
3. Initdb on all related hosts:

Cordinator
/usr/local/pgsql/bin/initdb -D /var/lib/postgres-xc/data --nodename cn1
DataNode1
/usr/local/pgsql/bin/initdb -D /var/lib/postgres-xc/data --nodename dn1
DataNode2
/usr/local/pgsql/bin/initdb -D /var/lib/postgres-xc/data --nodename dn2
GTM server
/usr/local/pgsql/bin/initgtm -Z gtm -D /var/lib/postgres-xc/data

Setting up the IP address of the GTM server on all nodes and start them with:
/usr/local/pgsql/bin/pg_ctl start -D /var/lib/postgres-xc/data -Z datanode -l /var/lib/postgres-xc/data/logfile
/usr/local/pgsql/bin/pg_ctl start -D /var/lib/postgres-xc/data -Z coordinator -l /var/lib/postgres-xc/data/logfile
/usr/local/pgsql/bin/gtm_ctl -Z gtm -D /var/lib/postgres-xc/data start
Adding nodes into cordinator:
postgres=# CREATE NODE dn1 WITH (HOST = '172.30.0.14', type = 'datanode', PORT = 5432);
postgres=# CREATE NODE dn2 WITH (HOST = '172.30.0.15', type = 'datanode', PORT = 5432);
postgres=# SELECT pgxc_pool_reload();

When I issue "createdb -h 172.30.0.11(cordinatorIP) -Upostgres-xc pgbench" from 5th node, it creates the DB on all nodes. Then I populate the DB with "pgbench -h 172.30.0.11 -i -s 10 -Upostgres-xc pgbench" and when i try to run any tests the whole setup crashes with:

starting vacuum...end.
starting vacuum pgbench_accounts...end.
Client 0 aborted in state 13: ERROR: failed to find GlobalTransaction for xid 10135
WARNING: unexpected EOF on datanode connection
Client 1 aborted in state 13: ERROR: sorry, too many clients already
transaction type: TPC-B (sort of)
scaling factor: 10
query mode: simple
number of clients: 2
number of threads: 2
duration: 3600 s
number of transactions actually processed: 126
tps = 42.505160 (including connections establishing)
tps = 42.612909 (excluding connections establishing)

On the coordinator node I am observing the folloging errors into log file:

LOG: failed to find proc 0x7fb0cb3f2480 in ProcArray
STATEMENT: drop table if exists pgbench_history
LOG: failed to find proc 0x7fb0cb3f2480 in ProcArray
STATEMENT: create table pgbench_history(tid int,bid int,aid int,delta int,mtime timestamp,filler char(22))
LOG: failed to find proc 0x7fb0cb3f2480 in ProcArray
STATEMENT: drop table if exists pgbench_tellers
LOG: failed to find proc 0x7fb0cb3f2480 in ProcArray
STATEMENT: create table pgbench_tellers(tid int not null,bid int,tbalance int,filler char(84)) with (fillfactor=100)
LOG: failed to find proc 0x7fb0cb3f2480 in ProcArray
STATEMENT: drop table if exists pgbench_accounts
LOG: failed to find proc 0x7fb0cb3f2480 in ProcArray
STATEMENT: create table pgbench_accounts(aid int not null,bid int,abalance int,filler char(84)) with (fillfactor=100)
LOG: failed to find proc 0x7fb0cb3f2480 in ProcArray
STATEMENT: drop table if exists pgbench_branches
LOG: failed to find proc 0x7fb0cb3f2480 in ProcArray
STATEMENT: create table pgbench_branches(bid int not null,bbalance int,filler char(88)) with (fillfactor=100)
LOG: failed to find proc 0x7fb0cb3f2480 in ProcArray
STATEMENT: commit
LOG: failed to find proc 0x7fb0cb3f2480 in ProcArray
STATEMENT: alter table pgbench_branches add primary key (bid)
LOG: failed to find proc 0x7fb0cb3f2480 in ProcArray
STATEMENT: alter table pgbench_tellers add primary key (tid)
LOG: failed to find proc 0x7fb0cb3f2480 in ProcArray
STATEMENT: alter table pgbench_accounts add primary key (aid)
LOG: failed to find proc 0x7fb0cb3f2480 in ProcArray
STATEMENT: truncate pgbench_history
ERROR: failed to find GlobalTransaction for xid 11378
STATEMENT: END;
WARNING: unexpected EOF on datanode connection
ERROR: sorry, too many clients already
STATEMENT: END;
WARNING: unexpected EOF on datanode connection
LOG: Failed to ABORT at node 16384
Detail: unexpected EOF on datanode connection
LOG: Failed to ABORT an implicitly PREPARED transaction status - 7
LOG: could not send data to client: Broken pipe
FATAL: connection to client lost
LOG: failed to find proc 0x7fb0cb34f7c0 in ProcArray
LOG: failed to find proc 0x7fb0cb34f7c0 in ProcArray
LOG: failed to find proc 0x7fb0cb34f7c0 in ProcArray
LOG: failed to find proc 0x7fb0cb34f7c0 in ProcArray
LOG: failed to find proc 0x7fb0cb34f7c0 in ProcArray**

On the data-nodes:

LOG: failed to find proc 0x7f379dee4a00 in ProcArray
LOG: failed to find proc 0x7f379dee4a00 in ProcArray
LOG: failed to find proc 0x7f379dee4a00 in ProcArray

and some of the transactons are still uncommited on some of the nodes and if I want to connect to other node ot throws the error "psql: FATAL: sorry, too many clients already"

Please any ideas, clue help....etc.

Thanks

Recent changes to 7: Postgres-XC crashes with failed to find proc 0x7fb0cb34f7c0 in ProcArray and sorry, too many clients already

#7 Postgres-XC crashes with failed to find proc 0x7fb0cb34f7c0 in ProcArray and sorry, too many clients already

#7 Postgres-XC crashes with failed to find proc 0x7fb0cb34f7c0 in ProcArray and sorry, too many clients already

Postgres-XC crashes with failed to find proc 0x7fb0cb34f7c0 in ProcArray and sorry, too many clients already