You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@phoenix.apache.org by "Anoop Sam John (JIRA)" <ji...@apache.org> on 2016/11/24 11:37:58 UTC
[jira] [Commented] (PHOENIX-3507) HBase secure wal log cash with
Phoenix secondary index
[ https://issues.apache.org/jira/browse/PHOENIX-3507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15693043#comment-15693043 ]
Anoop Sam John commented on PHOENIX-3507:
-----------------------------------------
Copied from HBase issue
"I believe I got what is the issue
AbstractProtobufLogWriter#initAfterHeader will do Codec create and will consider the config 'hbase.regionserver.wal.codec'. But SecureProtobufLogWriter, which is in use here, override initAfterHeader() and that in turn creates SecureWALCellCodec only. The even if the config says to use a diff Codec, that is not at all getting used. In Pheonix this Codec might be having some extra logic to handle some special KVs. There was an IndexKV usage in the past.. Not sure abt cur code base of Phoenix.
So HBase side issue is this - Config 'hbase.regionserver.wal.codec' is discarded when wal encryption is ON."
Ping [~apurtell].. With WAL encryption, we intended not to support custom wal cell codec? Or else we should make SecureWALCellCodec limited private and document that when encryption is ON, the custom wal codec impl has to extend this NOT WALCellCodec.
> HBase secure wal log cash with Phoenix secondary index
> ------------------------------------------------------
>
> Key: PHOENIX-3507
> URL: https://issues.apache.org/jira/browse/PHOENIX-3507
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 4.8.0
> Environment: Hadoop 2.7.3
> HBase 1.2.4
> Phoenix 4.8.1
> JDK 1.8.0_112
> Reporter: SonixLegend
> Fix For: 4.9.0
>
>
> When I used wal.encryption and phoenix secondary indexes, I got the error message
> 2016-11-24 17:11:20,361 INFO [regionserver/host-172-31-0-68/172.31.0.68:16020.logRoller] wal.FSHLog: Rolled WAL /hbase/WALs/host-172-31-0-68,16020,1479976404182/host-172-31-0-68%2C16020%2C1479976404182.default.1479978679455 with entries=0, filesize=242 B; new WAL /hbase/WALs/host-172-31-0-68,16020,1479976404182/host-172-31-0-68%2C16020%2C1479976404182.default.1479978680140
> 2016-11-24 17:11:20,363 INFO [regionserver/host-172-31-0-68/172.31.0.68:16020.logRoller] wal.FSHLog: Archiving hdfs://data/hbase/WALs/host-172-31-0-68,16020,1479976404182/host-172-31-0-68%2C16020%2C1479976404182.default.1479978679455 to hdfs://data/hbase/oldWALs/host-172-31-0-68%2C16020%2C1479976404182.default.1479978679455
> 2016-11-24 17:11:20,654 WARN [regionserver/host-172-31-0-68/172.31.0.68:16020.append-pool1-t1] wal.FSHLog: Append sequenceId=9, requesting roll of WAL
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.util.Bytes.toInt(Bytes.java:801)
> at org.apache.hadoop.hbase.util.Bytes.toInt(Bytes.java:788)
> at org.apache.hadoop.hbase.KeyValue.getKeyLength(KeyValue.java:1324)
> at org.apache.hadoop.hbase.KeyValue.getTagsLength(KeyValue.java:1638)
> at org.apache.hadoop.hbase.regionserver.wal.SecureWALCellCodec$EncryptedKvEncoder.write(SecureWALCellCodec.java:199)
> at org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter.append(ProtobufLogWriter.java:122)
> at org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.append(FSHLog.java:1932)
> at org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1794)
> at org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1704)
> at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> 2016-11-24 17:11:20,810 WARN [regionserver/host-172-31-0-68/172.31.0.68:16020.logRoller] wal.FSHLog: Failed sync-before-close but no outstanding appends; closing WAL: org.apache.hadoop.hbase.regionserver.wal.DamagedWALException: Append sequenceId=9, requesting roll of WAL
> And my hbase-site.xml is containing the settings.
> <property>
> <name>hbase.regionserver.wal.encryption</name>
> <value>true</value>
> </property>
> <property>
> <name>hbase.regionserver.hlog.reader.impl</name>
> <value>org.apache.hadoop.hbase.regionserver.wal.SecureProtobufLogReader</value>
> </property>
> <property>
> <name>hbase.regionserver.hlog.writer.impl</name>
> <value>org.apache.hadoop.hbase.regionserver.wal.SecureProtobufLogWriter</value>
> </property>
> <property>
> <name>hbase.regionserver.wal.codec</name>
> <value>org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec</value>
> </property>
> And my phoenix table sql
> CREATE SCHEMA TEST;
> CREATE TABLE TEST.TEST (ID BIGINT NOT NULL PRIMARY KEY, TEST VARCHAR);
> UPSERT INTO TEST.TEST (ID, TEST) VALUES (1, 'test');
> CREATE INDEX TEST_INDEX ON TEST.TEST (TEST);
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)