Home > oracle, Oracle Database, Weblogic > ‘Broken Pipe’ error in Application Server Logs While Connecting to the Database

‘Broken Pipe’ error in Application Server Logs While Connecting to the Database

February 24, 2012 Leave a comment Go to comments

I have encountered this problem at several customer sites, where the symptoms are the following:

In the application server logs the following error shows up while attempting a connection to the database:

Caused by: java.net.SocketException: Broken pipe
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:103)

In the database’s $ORACLE_HOME/network/log/sqlnet.log the following sequence shows up:

Fatal NI connect error 12170.

VERSION INFORMATION:
TNS for Linux: Version 10.2.0.4.0 – Production
Oracle Bequeath NT Protocol Adapter for Linux: Version 10.2.0.4.0 – Production
TCP/IP NT Protocol Adapter for Linux: Version 10.2.0.4.0 – Production
Time: 15-JAN-2012 17:09:30
Tracing not turned on.
Tns error struct:
ns main err code: 12535
TNS-12535: TNS:operation timed out
ns secondary err code: 12606
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
Client address: (ADDRESS=(PROTOCOL=tcp)(HOST=XXX.XXX.XXX.XXX)(PORT=1879))

The application server machine is quite slow and a large amount of CPU is consumed even though there is not much activity on the machine, and even a simple ‘java -version’ command might take up to minutes to return the output.

What actually happens is that the app server is able to reach the database and initiate a connection. The database will then wait for the app server to authenticate itself for a period of SQLNET.INBOUND_CONNECT_TIMEOUT seconds (by default 60 seconds). Since the app server will not be able to authenticate in time, the database will cut the connection due to the timeout, resulting in the ‘Broken Pipe’ error in the app server logs.

So far I’ve only encountered this problem on AIX machines, and it is quite difficult to diagnose it since all AS or Database settings are correct. The faulty behavior is actually caused by the way the operating system is resolving hostnames. So, if you are experiencing the above symptoms, you should check whether the operating system is using DNS to resolve hostnames. You can check this in the /etc/resolv.conf file. If you have any nameservers defined there, remove these lines and make sure you add whatever you need to resolve to the /etc/hosts file instead.

Advertisements
  1. No comments yet.
  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: