You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "helifu (Code Review)" <ge...@cloudera.org> on 2020/05/28 13:08:30 UTC
[kudu-CR] KUDU-1457: support for IPv6
helifu has uploaded this change for review. ( http://gerrit.cloudera.org:8080/15996
Change subject: KUDU-1457: support for IPv6
......................................................................
KUDU-1457: support for IPv6
This patch is intended to support IPv6 by expanding Sockaddr,
but please kindly note that webserver is not included.
Change-Id: Ie11e8e03152513284b7088dfba6c905a2fcdf46d
---
M src/kudu/rpc/reactor.cc
M src/kudu/rpc/rpc-test.cc
M src/kudu/util/net/net_util-test.cc
M src/kudu/util/net/net_util.cc
M src/kudu/util/net/net_util.h
M src/kudu/util/net/sockaddr.cc
M src/kudu/util/net/sockaddr.h
M src/kudu/util/net/socket-test.cc
M src/kudu/util/test_util.cc
9 files changed, 294 insertions(+), 104 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/96/15996/1
--
To view, visit http://gerrit.cloudera.org:8080/15996
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ie11e8e03152513284b7088dfba6c905a2fcdf46d
Gerrit-Change-Number: 15996
Gerrit-PatchSet: 1
Gerrit-Owner: helifu <he...@apache.org>
[kudu-CR] KUDU-1457 p1: krpc supports for IPv6
Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/15996 )
Change subject: KUDU-1457 p1: krpc supports for IPv6
......................................................................
Patch Set 2:
I suppose we need to enable ipv6 in our dist test environment. I think it might make sense to only run the ipv6 tests when ipv6 is enabled though regardless.
--
To view, visit http://gerrit.cloudera.org:8080/15996
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie11e8e03152513284b7088dfba6c905a2fcdf46d
Gerrit-Change-Number: 15996
Gerrit-PatchSet: 2
Gerrit-Owner: helifu <he...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: helifu <he...@apache.org>
Gerrit-Comment-Date: Fri, 29 May 2020 13:22:01 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-1457 p1: krpc supports for IPv6
Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/15996 )
Change subject: KUDU-1457 p1: krpc supports for IPv6
......................................................................
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/15996/2/src/kudu/util/net/net_util.cc
File src/kudu/util/net/net_util.cc:
http://gerrit.cloudera.org:8080/#/c/15996/2/src/kudu/util/net/net_util.cc@206
PS2, Line 206: hints.ai_family = AF_UNSPEC;
> yep, it's an incompatible change. But I'm afraid I can't find a better way
Is this something we can put behind a feature flag and leave defaulted to the old behavior so that users can opt into ipv6 support/behavior?
--
To view, visit http://gerrit.cloudera.org:8080/15996
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie11e8e03152513284b7088dfba6c905a2fcdf46d
Gerrit-Change-Number: 15996
Gerrit-PatchSet: 2
Gerrit-Owner: helifu <he...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: helifu <he...@apache.org>
Gerrit-Comment-Date: Fri, 05 Jun 2020 13:25:21 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-1457 p1: krpc supports for IPv6
Posted by "helifu (Code Review)" <ge...@cloudera.org>.
helifu has posted comments on this change. ( http://gerrit.cloudera.org:8080/15996 )
Change subject: KUDU-1457 p1: krpc supports for IPv6
......................................................................
Patch Set 2:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/15996/2/src/kudu/rpc/rpc-test.cc
File src/kudu/rpc/rpc-test.cc:
http://gerrit.cloudera.org:8080/#/c/15996/2/src/kudu/rpc/rpc-test.cc@136
PS2, Line 136: testing::Values(false, true),
> instead of a third bool, I think it makes more sense to change the bool to
Done
http://gerrit.cloudera.org:8080/#/c/15996/2/src/kudu/util/net/dns_resolver-test.cc
File src/kudu/util/net/dns_resolver-test.cc:
http://gerrit.cloudera.org:8080/#/c/15996/2/src/kudu/util/net/dns_resolver-test.cc@64
PS2, Line 64: ASSERT_TRUE(false)
> nit: FAIL()
Done
http://gerrit.cloudera.org:8080/#/c/15996/2/src/kudu/util/net/net_util.h
File src/kudu/util/net/net_util.h:
http://gerrit.cloudera.org:8080/#/c/15996/2/src/kudu/util/net/net_util.h@142
PS2, Line 142: Network(sa_family_t family, uint128_t addr, uint128_t netmask);
> does this make sense for ipv6? I dont know a lot about it but from googling
You mean "uint64" instead? If so, maybe we will lose some info which prevents address translation while calling "inet_ntop()".
1.IPv6: 8 * 16 = 128 bit.
struct sockaddr_in6 {
sa_family_t sin6_family; /* AF_INET6 */
in_port_t sin6_port; /* port number */
uint32_t sin6_flowinfo; /* IPv6 flow information */
struct in6_addr sin6_addr; /* IPv6 address */
uint32_t sin6_scope_id; /* Scope ID (new in 2.4) */
};
struct in6_addr {
unsigned char s6_addr[16]; /* IPv6 address */
};
2."/64" means prefix length.
http://gerrit.cloudera.org:8080/#/c/15996/2/src/kudu/util/net/net_util.cc
File src/kudu/util/net/net_util.cc:
http://gerrit.cloudera.org:8080/#/c/15996/2/src/kudu/util/net/net_util.cc@206
PS2, Line 206: hints.ai_family = AF_UNSPEC;
> will this be an incompatible change of sorts? do we have to worry that clie
yep, it's an incompatible change. But I'm afraid I can't find a better way than to specify a clear IPv4/IPv6 address(not hostname) for client. What do you think?
http://gerrit.cloudera.org:8080/#/c/15996/2/src/kudu/util/net/net_util.cc@232
PS2, Line 232: addr->sin6_scope_id = if_nametoindex(FLAGS_rpc_bind_interface_ipv6.c_str());
> what's this doing?
At first I just wanted to support IPv6 link-local address which requires scope_id, but now I think we could support it later:)
--
To view, visit http://gerrit.cloudera.org:8080/15996
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie11e8e03152513284b7088dfba6c905a2fcdf46d
Gerrit-Change-Number: 15996
Gerrit-PatchSet: 2
Gerrit-Owner: helifu <he...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: helifu <he...@apache.org>
Gerrit-Comment-Date: Fri, 05 Jun 2020 07:06:11 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-1457 p1: krpc supports for IPv6
Posted by "helifu (Code Review)" <ge...@cloudera.org>.
helifu has posted comments on this change. ( http://gerrit.cloudera.org:8080/15996 )
Change subject: KUDU-1457 p1: krpc supports for IPv6
......................................................................
Patch Set 2:
I did some tests on a VM box with 1 master + 3 tservers.
########################################################################################
[root@ecs-1458 release]# ./bin/kudu cluster ksck [::]:7051
Master Summary
UUID | Address | Status
----------------------------------+-----------+---------
b034de573adb437d92491766ffc04e02 | [::]:7051 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
-------------+-------+-----------------+-------------------------
never_fsync | true | unsafe,advanced | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------+---------+----------+----------------+-----------------
02fac19fd7a543d5afb640a62ccb5a9e | ecs-1458:31202 | HEALTHY | <none> | 2 | 0
437a5ec6d9314fc2b95b83e4e7b282da | ecs-1458:31201 | HEALTHY | <none> | 4 | 0
6da856c3230648cca17687d24ce5aff4 | ecs-1458:31203 | HEALTHY | <none> | 2 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
-------------+-------+-----------------+-------------------------
never_fsync | true | unsafe,advanced | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.13.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
--------------------+----+---------+---------------+---------+------------+------------------+-------------
impala::helf.part | 3 | HEALTHY | 4 | 4 | 0 | 0 | 0
impala::helf.part1 | 3 | HEALTHY | 4 | 4 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 8
First Quartile | 8
Median | 8
Third Quartile | 8
Maximum | 8
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 2
Tablets | 8
Replicas | 24
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
[root@ecs-1458 release]#
[root@ecs-1458 release]# ./bin/kudu perf loadgen [::]:7051
Using auto-created table 'default.loadgen_auto_218058633ff44beb9554b80621ae15fb'
INSERT report
rows total: 2000
time total: 12.2919 ms
time per row: 0.00614597 ms
Dropping auto-created table 'default.loadgen_auto_218058633ff44beb9554b80621ae15fb'
[root@ecs-1458 release]#
########################################################################################
[root@ecs-1458 impala]# impala-shell.sh
...
[localhost.localdomain:21000] helf> create table part1
> (
> P_PARTKEY BIGINT,
> P_NAME STRING,
> P_MFGR STRING,
> P_BRAND STRING,
> P_TYPE STRING,
> P_SIZE INT,
> P_CONTAINER STRING,
> P_RETAILPRICE DOUBLE,
> P_COMMENT STRING,
> primary key(P_PARTKEY)
> )
> PARTITION BY HASH(P_PARTKEY) PARTITIONS 4
> STORED AS KUDU
> TBLPROPERTIES ('kudu.master_addresses'='[::]:7051');
Query: create table part1
(
P_PARTKEY BIGINT,
P_NAME STRING,
P_MFGR STRING,
P_BRAND STRING,
P_TYPE STRING,
P_SIZE INT,
P_CONTAINER STRING,
P_RETAILPRICE DOUBLE,
P_COMMENT STRING,
primary key(P_PARTKEY)
)
PARTITION BY HASH(P_PARTKEY) PARTITIONS 4
STORED AS KUDU
TBLPROPERTIES ('kudu.master_addresses'='[::]:7051')
+-------------------------+
| summary |
+-------------------------+
| Table has been created. |
+-------------------------+
Fetched 1 row(s) in 0.38s
[localhost.localdomain:21000] helf>
[localhost.localdomain:21000] helf>
[localhost.localdomain:21000] helf>
[localhost.localdomain:21000] helf> insert into part1 select * from tpch.part;
Query: insert into part1 select * from tpch.part
Query submitted at: 2020-06-05 15:58:02 (Coordinator: http://ecs-1458:25000)
Query progress can be monitored at: http://ecs-1458:25000/query_plan?query_id=4940ca92f5fa46b7:8b2a0a3300000000
Modified 200000 row(s), 0 row error(s) in 8.45s
[localhost.localdomain:21000] helf>
[localhost.localdomain:21000] helf> select count(*) from part1;
Query: select count(*) from part1
Query submitted at: 2020-06-05 15:58:29 (Coordinator: http://ecs-1458:25000)
Query progress can be monitored at: http://ecs-1458:25000/query_plan?query_id=114cc951c86a681a:12f7eeb200000000
+----------+
| count(*) |
+----------+
| 200000 |
+----------+
Fetched 1 row(s) in 0.12s
[localhost.localdomain:21000] helf> select * from part1 limit 10;
Query: select * from part1 limit 10
Query submitted at: 2020-06-05 15:58:35 (Coordinator: http://ecs-1458:25000)
Query progress can be monitored at: http://ecs-1458:25000/query_plan?query_id=7147f3e0b28934e9:4b380abe00000000
+-----------+-----------------------------------------+----------------+----------+-------------------------+--------+-------------+---------------+-----------------------+
| p_partkey | p_name | p_mfgr | p_brand | p_type | p_size | p_container | p_retailprice | p_comment |
+-----------+-----------------------------------------+----------------+----------+-------------------------+--------+-------------+---------------+-----------------------+
| 2 | blush thistle blue yellow saddle | Manufacturer#1 | Brand#13 | LARGE BRUSHED BRASS | 1 | LG CASE | 902 | lar accounts amo |
| 3 | spring green yellow purple cornsilk | Manufacturer#4 | Brand#42 | STANDARD POLISHED BRASS | 21 | WRAP CASE | 903 | egular deposits hag |
| 13 | ghost olive orange rosy thistle | Manufacturer#5 | Brand#55 | MEDIUM BURNISHED NICKEL | 1 | JUMBO PACK | 913.01 | osits. |
| 16 | deep sky turquoise drab peach | Manufacturer#3 | Brand#32 | PROMO PLATED TIN | 2 | MED PACK | 916.01 | unts a |
| 20 | ivory navy honeydew sandy midnight | Manufacturer#1 | Brand#12 | LARGE POLISHED NICKEL | 48 | MED BAG | 920.02 | are across the asympt |
| 24 | seashell coral metallic midnight floral | Manufacturer#5 | Brand#52 | MEDIUM PLATED STEEL | 20 | MED CASE | 924.02 | final the |
| 26 | beige frosted moccasin chocolate snow | Manufacturer#3 | Brand#32 | SMALL BRUSHED STEEL | 32 | SM CASE | 926.02 | instructions i |
| 29 | lemon sky grey salmon orchid | Manufacturer#3 | Brand#33 | PROMO PLATED COPPER | 7 | LG DRUM | 929.02 | carefully fluffi |
| 30 | cream misty steel spring medium | Manufacturer#4 | Brand#42 | PROMO ANODIZED TIN | 17 | LG BOX | 930.03 | carefully bus |
| 34 | khaki steel rose ghost salmon | Manufacturer#1 | Brand#13 | LARGE BRUSHED STEEL | 8 | JUMBO BOX | 934.03 | riously ironic |
+-----------+-----------------------------------------+----------------+----------+-------------------------+--------+-------------+---------------+-----------------------+
Fetched 10 row(s) in 0.11s
[localhost.localdomain:21000] helf>
--
To view, visit http://gerrit.cloudera.org:8080/15996
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie11e8e03152513284b7088dfba6c905a2fcdf46d
Gerrit-Change-Number: 15996
Gerrit-PatchSet: 2
Gerrit-Owner: helifu <he...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: helifu <he...@apache.org>
Gerrit-Comment-Date: Fri, 05 Jun 2020 08:12:52 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-1457 p1: krpc supports for IPv6
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/15996 )
Change subject: KUDU-1457 p1: krpc supports for IPv6
......................................................................
Patch Set 2:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/15996/2/src/kudu/rpc/rpc-test.cc
File src/kudu/rpc/rpc-test.cc:
http://gerrit.cloudera.org:8080/#/c/15996/2/src/kudu/rpc/rpc-test.cc@136
PS2, Line 136: testing::Values(false, true),
instead of a third bool, I think it makes more sense to change the bool to an enum with three options (doesn't make sense to be both use_unix=true and use_ipv6=true)
http://gerrit.cloudera.org:8080/#/c/15996/2/src/kudu/util/net/dns_resolver-test.cc
File src/kudu/util/net/dns_resolver-test.cc:
http://gerrit.cloudera.org:8080/#/c/15996/2/src/kudu/util/net/dns_resolver-test.cc@64
PS2, Line 64: ASSERT_TRUE(false)
nit: FAIL()
http://gerrit.cloudera.org:8080/#/c/15996/2/src/kudu/util/net/net_util.h
File src/kudu/util/net/net_util.h:
http://gerrit.cloudera.org:8080/#/c/15996/2/src/kudu/util/net/net_util.h@142
PS2, Line 142: Network(sa_family_t family, uint128_t addr, uint128_t netmask);
does this make sense for ipv6? I dont know a lot about it but from googling it sounds like all IPV6 addresses are /64
http://gerrit.cloudera.org:8080/#/c/15996/2/src/kudu/util/net/net_util.cc
File src/kudu/util/net/net_util.cc:
http://gerrit.cloudera.org:8080/#/c/15996/2/src/kudu/util/net/net_util.cc@206
PS2, Line 206: hints.ai_family = AF_UNSPEC;
will this be an incompatible change of sorts? do we have to worry that clients will start resolving IPV6 endpoints for clusters which are multihomed on both ipv4 and ipv6, and stop working since now the client will talk to an ipv6 address that isn't actually listening? (old server)
http://gerrit.cloudera.org:8080/#/c/15996/2/src/kudu/util/net/net_util.cc@232
PS2, Line 232: addr->sin6_scope_id = if_nametoindex(FLAGS_rpc_bind_interface_ipv6.c_str());
what's this doing?
--
To view, visit http://gerrit.cloudera.org:8080/15996
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie11e8e03152513284b7088dfba6c905a2fcdf46d
Gerrit-Change-Number: 15996
Gerrit-PatchSet: 2
Gerrit-Owner: helifu <he...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: helifu <he...@apache.org>
Gerrit-Comment-Date: Tue, 02 Jun 2020 05:40:11 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-1457 p1: krpc supports for IPv6
Posted by "helifu (Code Review)" <ge...@cloudera.org>.
Hello Alexey Serbin, Kudu Jenkins, Andrew Wong, Grant Henke, Todd Lipcon,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/15996
to look at the new patch set (#3).
Change subject: KUDU-1457 p1: krpc supports for IPv6
......................................................................
KUDU-1457 p1: krpc supports for IPv6
This patch is intended to support IPv6 by expanding
Sockaddr, but please kindly note that:
1) webserver is not included yet;
2) IPv6 link-local is not supported;
Change-Id: Ie11e8e03152513284b7088dfba6c905a2fcdf46d
---
M src/kudu/rpc/reactor.cc
M src/kudu/rpc/rpc-test.cc
M src/kudu/util/net/dns_resolver-test.cc
M src/kudu/util/net/net_util-test.cc
M src/kudu/util/net/net_util.cc
M src/kudu/util/net/net_util.h
M src/kudu/util/net/sockaddr.cc
M src/kudu/util/net/sockaddr.h
M src/kudu/util/net/socket-test.cc
M src/kudu/util/test_util.cc
10 files changed, 334 insertions(+), 115 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/96/15996/3
--
To view, visit http://gerrit.cloudera.org:8080/15996
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie11e8e03152513284b7088dfba6c905a2fcdf46d
Gerrit-Change-Number: 15996
Gerrit-PatchSet: 3
Gerrit-Owner: helifu <he...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: helifu <he...@apache.org>
[kudu-CR] KUDU-1457 p1: krpc supports for IPv6
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/15996 )
Change subject: KUDU-1457 p1: krpc supports for IPv6
......................................................................
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/15996/2/src/kudu/util/net/net_util.h
File src/kudu/util/net/net_util.h:
http://gerrit.cloudera.org:8080/#/c/15996/2/src/kudu/util/net/net_util.h@142
PS2, Line 142: Network(sa_family_t family, uint128_t addr, uint128_t netmask);
> You mean "uint64" instead? If so, maybe we will lose some info which preven
I think this Network class is only used for our whitelisting of which subnets are allowed to connect when authentication hasn't been enabled. But, in ipv6, I think all subnets are /64 if I understand correctly, so maybe we don't need to use this class at all?
--
To view, visit http://gerrit.cloudera.org:8080/15996
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie11e8e03152513284b7088dfba6c905a2fcdf46d
Gerrit-Change-Number: 15996
Gerrit-PatchSet: 2
Gerrit-Owner: helifu <he...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: helifu <he...@apache.org>
Gerrit-Comment-Date: Fri, 05 Jun 2020 14:34:51 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-1457 p1: krpc supports for IPv6
Posted by "helifu (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/15996
to look at the new patch set (#2).
Change subject: KUDU-1457 p1: krpc supports for IPv6
......................................................................
KUDU-1457 p1: krpc supports for IPv6
This patch is intended to support IPv6 by expanding Sockaddr,
but please kindly note that webserver is not included yet.
Change-Id: Ie11e8e03152513284b7088dfba6c905a2fcdf46d
---
M src/kudu/rpc/reactor.cc
M src/kudu/rpc/rpc-test.cc
M src/kudu/util/net/dns_resolver-test.cc
M src/kudu/util/net/net_util-test.cc
M src/kudu/util/net/net_util.cc
M src/kudu/util/net/net_util.h
M src/kudu/util/net/sockaddr.cc
M src/kudu/util/net/sockaddr.h
M src/kudu/util/net/socket-test.cc
M src/kudu/util/test_util.cc
10 files changed, 305 insertions(+), 106 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/96/15996/2
--
To view, visit http://gerrit.cloudera.org:8080/15996
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie11e8e03152513284b7088dfba6c905a2fcdf46d
Gerrit-Change-Number: 15996
Gerrit-PatchSet: 2
Gerrit-Owner: helifu <he...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-1457 p1: krpc supports for IPv6
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/15996 )
Change subject: KUDU-1457 p1: krpc supports for IPv6
......................................................................
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/15996/2/src/kudu/util/net/net_util.cc
File src/kudu/util/net/net_util.cc:
http://gerrit.cloudera.org:8080/#/c/15996/2/src/kudu/util/net/net_util.cc@206
PS2, Line 206: hints.ai_family = AF_UNSPEC;
> I'm afraid not. For example, if the user opts into ipv6 but the real addres
Could we somehow pass information on what address family we are interested in when working with this function?
For the server side (i.e. kudu-master and kudu-tserver), we could specify what IP addresses to use by some new gflag: IPv4 and IPv6, IPv4 only, IPv6 only. That will affect what type of sockets the server should bind and listen to: both IPv4 and IPv6, IPv4 only, IPv6 only correspondingly.
For the client side, we could deduce the type of sockets to work based on the way how master RPC end-points are specified:
* if they are given as IPv4 addresses/ports, then use IPv4 only addresses
* if they are given as IPv6 addresses/ports, then use IPv6 only addresses
* if they are given as DNS names/ports, then client can resolve those providing 'IPv6 and IPv4' preference, and then try to connect. If both IPv6 and IPv4 addresses are served by Kudu masters, then use IPv6, otherwise use IPv4 for all further communications with the cluster.
For the client side, another option might be relying on some environment variable, say KUDU_USE_IPV6 or something. Not the best choice, but at this point we don't have other ways to provide configuration for Kudu client library as a part of an application.
--
To view, visit http://gerrit.cloudera.org:8080/15996
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie11e8e03152513284b7088dfba6c905a2fcdf46d
Gerrit-Change-Number: 15996
Gerrit-PatchSet: 2
Gerrit-Owner: helifu <he...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: helifu <he...@apache.org>
Gerrit-Comment-Date: Thu, 18 Jun 2020 01:34:46 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-1457 p1: krpc supports for IPv6
Posted by "helifu (Code Review)" <ge...@cloudera.org>.
helifu has posted comments on this change. ( http://gerrit.cloudera.org:8080/15996 )
Change subject: KUDU-1457 p1: krpc supports for IPv6
......................................................................
Patch Set 3:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/15996/2/src/kudu/util/net/net_util.h
File src/kudu/util/net/net_util.h:
http://gerrit.cloudera.org:8080/#/c/15996/2/src/kudu/util/net/net_util.h@142
PS2, Line 142: Network(sa_family_t family, uint128_t addr, uint128_t netmask);
> I think this Network class is only used for our whitelisting of which subne
Thanks for your suggests, but let me clarify it if I understand correctly: ipv4 still uses the Network class, but ipv6 uses an uint64 instead? Or parent/child inheritance class: class Network --> class NetworkIPv4/class NetworkIPv6/ In addition, should we use another flag to indicate trusted ipv6 subnets?
http://gerrit.cloudera.org:8080/#/c/15996/2/src/kudu/util/net/net_util.cc
File src/kudu/util/net/net_util.cc:
http://gerrit.cloudera.org:8080/#/c/15996/2/src/kudu/util/net/net_util.cc@206
PS2, Line 206: RETURN_NOT_OK(GetAddrInfo(host_, hints, op_description, &result));
> Is this something we can put behind a feature flag and leave defaulted to t
I'm afraid not. For example, if the user opts into ipv6 but the real address(host_) is ipv4, then the operation will fail.
--
To view, visit http://gerrit.cloudera.org:8080/15996
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie11e8e03152513284b7088dfba6c905a2fcdf46d
Gerrit-Change-Number: 15996
Gerrit-PatchSet: 3
Gerrit-Owner: helifu <he...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: helifu <he...@apache.org>
Gerrit-Comment-Date: Mon, 08 Jun 2020 06:16:34 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-1457 p1: krpc supports for IPv6
Posted by "Attila Bukor (Code Review)" <ge...@cloudera.org>.
Attila Bukor has posted comments on this change. ( http://gerrit.cloudera.org:8080/15996 )
Change subject: KUDU-1457 p1: krpc supports for IPv6
......................................................................
Patch Set 3:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/15996/3/src/kudu/util/net/net_util.h
File src/kudu/util/net/net_util.h:
http://gerrit.cloudera.org:8080/#/c/15996/3/src/kudu/util/net/net_util.h@104
PS3, Line 104: // Returns true if addr is within "127.0.0.0/8" or "::1" range.
nit: as ::1 is not a range, maybe rephrase to "127.0.0.0/8" range or "::1"
http://gerrit.cloudera.org:8080/#/c/15996/3/src/kudu/util/net/net_util.h@107
PS3, Line 107: // Returns a common representation of IP address in addr.
nit: add both IPv4 and IPv6 example
http://gerrit.cloudera.org:8080/#/c/15996/3/src/kudu/util/net/sockaddr.h
File src/kudu/util/net/sockaddr.h:
http://gerrit.cloudera.org:8080/#/c/15996/3/src/kudu/util/net/sockaddr.h@154
PS3, Line 154: bool is_ip() const {
does it make sense to add is_ipv4() and is_ipv6() methods as well?
http://gerrit.cloudera.org:8080/#/c/15996/3/src/kudu/util/test_util.cc
File src/kudu/util/test_util.cc:
http://gerrit.cloudera.org:8080/#/c/15996/3/src/kudu/util/test_util.cc@497
PS3, Line 497: return WaitForBind(pid, port, addr, "4UDP", timeout);
shouldn't this be changed to UDP as well?
--
To view, visit http://gerrit.cloudera.org:8080/15996
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie11e8e03152513284b7088dfba6c905a2fcdf46d
Gerrit-Change-Number: 15996
Gerrit-PatchSet: 3
Gerrit-Owner: helifu <he...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: helifu <he...@apache.org>
Gerrit-Comment-Date: Fri, 10 Jul 2020 13:56:54 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-1457 p1: krpc supports for IPv6
Posted by "helifu (Code Review)" <ge...@cloudera.org>.
helifu has posted comments on this change. ( http://gerrit.cloudera.org:8080/15996 )
Change subject: KUDU-1457 p1: krpc supports for IPv6
......................................................................
Patch Set 2:
> Build Started http://jenkins.kudu.apache.org/job/kudu-gerrit/21755/
It seems that the ipv6 is disabled since most to the failures are:
Bad status: Network error: connect(2) error: Cannot assign requested address (error 99)
But how can we enable the ipv6 while running unit tests?
echo 0 > /proc/sys/net/ipv6/conf/all/disable_ipv6
--
To view, visit http://gerrit.cloudera.org:8080/15996
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie11e8e03152513284b7088dfba6c905a2fcdf46d
Gerrit-Change-Number: 15996
Gerrit-PatchSet: 2
Gerrit-Owner: helifu <he...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: helifu <he...@apache.org>
Gerrit-Comment-Date: Fri, 29 May 2020 07:34:37 +0000
Gerrit-HasComments: No