You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hawq.apache.org by "Gagan Brahmi (JIRA)" <ji...@apache.org> on 2016/05/04 02:49:12 UTC
[jira] [Commented] (HAWQ-714) HAWQ can specify ip address used for
a node
[ https://issues.apache.org/jira/browse/HAWQ-714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15269946#comment-15269946 ]
Gagan Brahmi commented on HAWQ-714:
-----------------------------------
Attaching additional logs from the segment node. Hope this helps to get more details for a potential solution.
{quote}
2016-05-03 12:47:24.970195 MST,,,p45609,th-492025632,,,,0,,,seg-10000,,,,,"LOG","00000","HAWQ :: Received signal notification that HAWQ RM works now.",,,,,,,0,,"resourcemanager.c",425,
2016-05-03 12:47:24.970201 MST,,,p45609,th-492025632,,,,0,,,seg-10000,,,,,"LOG","00000","PostgreSQL 8.2.15 (Greenplum Database 4.2.0 build 1) (HAWQ 2.0.0.0_beta build dev) on x86_64-unknown-linux-gnu, com
piled by GCC gcc (GCC) 4.7.4 compiled on Mar 4 2016 04:48:40",,,,,,,0,,"postmaster.c",3694,
2016-05-03 12:47:24.970207 MST,,,p45609,th-492025632,,,,0,,,seg-10000,,,,,"LOG","00000","database system is ready to accept connections","PostgreSQL 8.2.15 (Greenplum Database 4.2.0 build 1) (HAWQ 2.0.0.0
_beta build dev) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.7.4 compiled on Mar 4 2016 04:48:40",,,,,,0,,"postmaster.c",3701,
2016-05-03 12:47:24.970231 MST,,,p45617,th-492025632,,,,0,,,seg-10000,,,,,"LOG","00000","HAWQ RM SEG process works now.",,,,,,,0,,"resourcemanager_RMSEG.c",72,
2016-05-03 12:47:25.070698 MST,,,p45617,th-492025632,,,,0,,,seg-10000,,,,,"LOG","00000","Resource manager discovered local host IPv4 address 127.0.0.1",,,,,,,0,,"network_utils.c",201,
2016-05-03 12:47:25.070775 MST,,,p45617,th-492025632,,,,0,,,seg-10000,,,,,"LOG","00000","Resource manager discovered local host IPv4 address 127.0.0.2",,,,,,,0,,"network_utils.c",201,
2016-05-03 12:47:25.070830 MST,,,p45617,th-492025632,,,,0,,,seg-10000,,,,,"LOG","00000","Resource manager discovered local host IPv4 address 192.168.0.18",,,,,,,0,,"network_utils.c",201,
2016-05-03 12:47:25.070866 MST,,,p45617,th-492025632,,,,0,,,seg-10000,,,,,"LOG","00000","Resource manager discovered local host IPv4 address 192.168.59.104",,,,,,,0,,"network_utils.c",201,
2016-05-03 12:47:25.070910 MST,,,p45617,th-492025632,,,,0,,,seg-10000,,,,,"LOG","00000","Segment resource manager discovers localhost configuration the first time.",,,,,,,0,,"requesthandler_RMSEG.c",104,
2016-05-03 12:47:25.070944 MST,,,p45617,th-492025632,,,,0,,,seg-10000,,,,,"LOG","00000","Segment resource manager Build/update local host information.",,,,,,,0,,"requesthandler_RMSEG.c",174,
2016-05-03 12:47:25.070997 MST,,,p45617,th-492025632,,,,0,,,seg-10000,,,,,"LOG","00000","Built localhost information. NODE:ID=-1,HAWQ AVAIL, GRM UNAVAIL, HAWQ CAP (65536 MB, 16.000000 CORE), GRM CAP(0 MB,
0.000000 CORE),NODE:HOST=suse11-hdp-node-1.gagan-lab.com:40000,Master:0,Standby:0,Alive:1.Addresses:127.0.0.1,127.0.0.2,192.168.0.18,192.168.59.104.",,,,,,,0,,"requesthandler_RMSEG.c",268,
2016-05-03 12:47:25.072056 MST,,,p45617,th-492025632,,,,0,,,seg-10000,,,,,"LOG","00000","checkAndBuildFailedTmpDirList finished checking temporary directory, which costs 16 us",,,,,,,0,,"resourcemanager_R
MSEG.c",290,
2016-05-03 12:47:55.128799 MST,,,p45617,th-492025632,,,,0,,,seg-10000,,,,,"LOG","00000","Resource manager discovered local host IPv4 address 127.0.0.1",,,,,,,0,,"network_utils.c",201,
2016-05-03 12:47:55.128835 MST,,,p45617,th-492025632,,,,0,,,seg-10000,,,,,"LOG","00000","Resource manager discovered local host IPv4 address 127.0.0.2",,,,,,,0,,"network_utils.c",201,
2016-05-03 12:47:55.128844 MST,,,p45617,th-492025632,,,,0,,,seg-10000,,,,,"LOG","00000","Resource manager discovered local host IPv4 address 192.168.0.18",,,,,,,0,,"network_utils.c",201,
2016-05-03 12:47:55.128851 MST,,,p45617,th-492025632,,,,0,,,seg-10000,,,,,"LOG","00000","Resource manager discovered local host IPv4 address 192.168.59.104",,,,,,,0,,"network_utils.c",201,
{quote}
In case of this node, the operating system had additional IP address attached to lo interface.
> HAWQ can specify ip address used for a node
> -------------------------------------------
>
> Key: HAWQ-714
> URL: https://issues.apache.org/jira/browse/HAWQ-714
> Project: Apache HAWQ
> Issue Type: Improvement
> Components: Fault Tolerance
> Reporter: Lei Chang
> Assignee: Lei Chang
>
> no, you cannot. hawq will take all the ip addresses from all the nodes.
> currently it is required that all the nodes cannot have any identical ip address. It is an area that can be improved though.
> Thanks
> Lei
> On Wed, May 4, 2016 at 6:00 AM, Gagan Brahmi <ga...@gmail.com> wrote:
> Thank you Vineet.
> I figured this problem out in the virtual environment. However, in the
> physical nodes this problem seemed to have caused due to an alias for
> loopback interface. The IP address which was causing the problem was
> 127.0.0.2.
> This brings me to another question. Is there any way we can configure
> HAWQ any specific IP address. In this case can we ask HAWQ to skip
> 127.0.0.2.
> Regards,
> Gagan Brahmi
> On Tue, May 3, 2016 at 11:30 AM, Vineet Goel <vg...@pivotal.io> wrote:
> > Are you using vagrant or VMs, or are these physical machines?
> >
> > Sometimes, the problem can result from the misleading IP address configuration of network card in virtual machine. Check if two segments have the same IP address in eth0.
> > You must specify different IP address of eth0 of different VMs.
> >
> > Thanks
> > -Vineet
> >
> >
> >
> > On May 3, 2016, at 8:30 AM, Gagan Brahmi <ga...@gmail.com> wrote:
> >
> > Hi All,
> >
> > I was looking to check if anyone has seen this behavior where segments
> > are not able to communicate with the master in HAWQ 2.0.
> >
> > In a single node setup I don't see any problem with the segment and
> > master communication. The problem seems to be visible if there is two
> > or three machine involved in the hawq cluster.
> >
> > The gp_segment_configuration reports the segment. It is reported up
> > for a few seconds and then it turns it down. If you execute any
> > queries the segment is no longer found in the
> > gp_segment_configuration.
> >
> > Nothing can be found in gp_configuration or gp_configuration_history.
> > hawqstate reports "failures at master" for the segments.
> >
> > I found a jira which had pretty much similar behavior (except for core
> > dumps which I haven't seen yet. Since I am not able run any queries.).
> > The jira in question is https://issues.apache.org/jira/browse/HAWQ-323
> >
> > This issue was closed stating duplicate IP. I am trying to understand
> > if that can be the case.
> >
> > There is no firewall between the segments. Nothing in blocking port
> > 40000, 5432, 5437 or 5438. Segment starts up fine. psql to segments on
> > port 40000 works fine as well.
> >
> > There is no error in the segments or master logs or startup logs.
> >
> > I tried to integrate hawq with YARN and also using it's own
> > ResourceManager, but found similar behavior. Tried to set the
> > heartbeat interval to 10 seconds (hawq_rm_segment_heartbeat_interval =
> > 10 in hawq-site.xml) but no change in the behavior.
> >
> > Am I missing anything here? Has anyone found similar behavior before?
> >
> >
> > Regards,
> > Gagan Brahmi
> >
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)