You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "Dmitry (JIRA)" <ji...@apache.org> on 2017/11/15 13:41:01 UTC
[jira] [Commented] (KYLIN-2849) duplicate segment，cannot be deleted and data cannot be refreshed and merged
[ https://issues.apache.org/jira/browse/KYLIN-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16253450#comment-16253450 ]
Dmitry commented on KYLIN-2849:
-------------------------------
Duplicate segments can appear when something wrong happens with job. In our case there were a job error. And after its discarding we got two segments with the same names. The second segment has status = NEW:
{
"uuid": "4a6f9098-d95c-4cca-8865-6efe2fc1e2f8",
"name": "20171030000000_20171211000000",
"storage_location_identifier": "KYLIN_VTYLURM5N4",
"date_range_start": 1509321600000,
"date_range_end": 1512950400000,
"source_offset_start": 0,
"source_offset_end": 0,
"status": "NEW",
"size_kb": 0,
"input_records": 0,
"input_records_size": 0,
"last_build_time": 0,
"last_build_job_id": null,
"create_time_utc": 1510412403197,
"cuboid_shard_nums": {},
"total_shards": 0,
"blackout_cuboids": [],
"binary_signature": null,
"dictionaries": null,
"snapshots": null,
"rowkey_stats": []
}
And when we try to delete it, we get an error:
{{"msg": "Cannot delete segment '20171030000000_20171211000000' as its status is not READY. Discard the on-going job for it.".}}
But there are no any jobs running. And there are no HBase table called KYLIN_VTYLURM5N4. Also it's not possible to REFRESH this segment. It says about Segments overlapping.
I guess it's related to KYLIN-2962 issue.
It would be great to be able to delete such segments from GUI or via RestfulAPI.
Billy Liu, what data should be removed from metadata to fix this issue?
> duplicate segment，cannot be deleted and data cannot be refreshed and merged
> ---------------------------------------------------------------------------
>
> Key: KYLIN-2849
> URL: https://issues.apache.org/jira/browse/KYLIN-2849
> Project: Kylin
> Issue Type: Bug
> Components: Job Engine, Metadata, REST Service
> Affects Versions: v2.0.0
> Environment: hadoop：hadoop-2.6.0-cdh5.8.2
> hive ：2.1.0
> hbase：0.98
> Reporter: 翟玉勇
> Assignee: Dong Li
> Attachments: kylin-1.png, kylin-2.png
>
>
> cube duplicate segments。
> cannot be deleted and data cannot be refreshed and merged
> {code}
> try
> curl -X DELETE "http://127.0.0.1:7070/kylin/api/cubes/Remain_Cube_2/segs/20170822000000_20170823000000" -H "Authorization: Basic QURNSU46S1lMSU4=" -H "Content-Type: application/json;charset=UTF-8"
> Cannot delete segment '20170822000000_20170823000000' as it is neither the first nor the last segment.","stacktrace":"org.apache.kylin.rest.exception.InternalErrorException: Cannot delete segment '20170822000000_20170823000000' as it is neither the first nor the last segment
> {code}
> 暂时解决办法：
> {code}
> public CubeInstance deleteSegment(CubeInstance cube, String segmentName) throws IOException {
> if (!segmentName.equals(cube.getSegments().get(0).getName()) && !segmentName.equals(cube.getSegments().get(cube.getSegments().size() - 1).getName())) {
> //throw new IllegalArgumentException("Cannot delete segment '" + segmentName + "' as it is neither the first nor the last segment.");
> }
> CubeSegment toDelete = null;
> for (CubeSegment seg : cube.getSegments()) {
> if (seg.getName().equals(segmentName)) {
> toDelete = seg;
> }
> }
> if (toDelete == null) {
> throw new IllegalArgumentException("Cannot find segment '" + segmentName + "'");
> }
> if (toDelete.getStatus() != SegmentStatusEnum.READY) {
> //throw new IllegalArgumentException("Cannot delete segment '" + segmentName + "' as its status is not READY. Discard the on-going job for it.");
> }
> CubeUpdate update = new CubeUpdate(cube);
> update.setToRemoveSegs(new CubeSegment[] { toDelete });
> return CubeManager.getInstance(getConfig()).updateCube(update);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)