You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@asterixdb.apache.org by "Michael J. Carey (JIRA)" <ji...@apache.org> on 2019/08/08 22:02:00 UTC
[jira] [Created] (ASTERIXDB-2627) internal error during LOAD (found
by Nick)
Michael J. Carey created ASTERIXDB-2627:
-------------------------------------------
Summary: internal error during LOAD (found by Nick)
Key: ASTERIXDB-2627
URL: https://issues.apache.org/jira/browse/ASTERIXDB-2627
Project: Apache AsterixDB
Issue Type: Improvement
Components: *DB - AsterixDB, FAIL - Failure handling/reporting, ING - Ingestion, RT - Runtime
Affects Versions: 0.9.4.1, 0.9.5
Reporter: Michael J. Carey
Assignee: Ian Maxon
Fix For: 0.9.5
Attachments: MASTER-personicle-espresso-schema-configuration-script.sqlpp, sample-event-dump.jsonl
*RUN THIS:*
USE Personicle;
//dropping and recreating since AsterixDB requires a virgin
DROP DATASET events IF EXISTS;
CREATE DATASET events(EventType) PRIMARY KEY eventId AUTOGENERATED;
LOAD DATASET events USING localfs
(("path"=":///Users/macandcheese/Desktop/Personicle/old-personicle-parser/sample-event-dump.jsonl"),("format"="adm"));
*BUT FIRST RUN THIS TO SET THINGS UP:*
--To begin we’ll go ahead and delete any existing dataverse named Personicle
--this ensures we’re writing into an empty dataverse (note ALL data in the
--dataverse will be deleted, so be careful). This is included so that this
--code can be re-run without causing naming conflicts, but is not required
--to create a dataverse.
DROP DATAVERSE Personicle IF EXISTS;
--Now that we have a clean slate let’s create our dataverse and call it
--Personicle, though the name is arbitrary.
CREATE DATAVERSE Personicle;
--Sets the current working dataverse for whatever SQL++ statements follow it.
--This is akin to USE in SQL.
USE Personicle;
--The question mark represents an optional field which may or may not be present.
--I’ve also specified some uuid types which I’ll later use for some auto-generated
--primary keys. Finally, ADM supports types within types, so you can use custom
--types for the fields. I’m not going to delve too deep into this, because I think the
--documentation explains it sufficiently more or less. Here are a couple links to
--get you started:
--https://ci.apache.org/projects/asterixdb/datamodel.html
--https://ci.apache.org/projects/asterixdb/sqlpp/primer-sqlpp.html
CREATE TYPE AddressType AS OPEN {
category: string?,
address: string,
city: string,
state: string,
postalCode: string
};
CREATE TYPE PhoneDetailType AS OPEN {
category: string?,
phone: string
};
CREATE TYPE EmailDetailType AS OPEN {
category: string?,
email: string
};
CREATE TYPE UserType AS OPEN {
--primary key uuid for user
userId: uuid,
name: string?,
dateOfBirth: date?,
addresses: [AddressType]?,
phones: [PhoneDetailType]?,
emails: [EmailDetailType]?
};
CREATE TYPE ObservationType AS OPEN {
observationId: uuid, --primary key uuid for observation
dataSource: uuid, --foreign key to the data source which generated the observation, like a smart device, API or human.
about: uuid, --foreign key to the user this observation is about
label: string?, --labels are human readable identifiers, for example "location observation for James"
start_at: datetime, --when the observation began being observed
end_at: datetime? --when the observation stopped being observed, some observations won't need this
};
CREATE TYPE EventType AS OPEN {
eventId: uuid, --primary key uuid for event
participant: uuid, --foreign key to user that participated in this event
start_at: datetime, --start of event
end_at: datetime, --end of event using `` because of a potential AsterixDB bug
name: string,
level: bigint?, --Jordan's paper referred to different levels of daily activities.
observations: [uuid] --foreign keys to ids of observations which were used to make up this event
};
CREATE TYPE DataSourceType AS OPEN {
dataSourceId: uuid, --primary key uuid for data sources
owner: uuid, --foreign key to the user who owns this data source
label: string?, --human-readable string for the data source, example: "Mike's iPhone"
personicleHost: string? --this represents metadata about the Personicle application which was used for this data source, example: "Personicle Android application v1.2.43"
};
--The code below creates collections using the ADM specified above. --Therefore this code can be run as boilerplate code to get started with using AsterixDB.
--Note that I've specified the primary key to be "AUTOGENERATED" which
--means AsterixDB will create one for us if the UUID key is omitted,
--if it is provided AsterixDB will use the UUID that was provided.
USE Personicle;
--We will have one collection for each of the ER diagram's top-level entity type instances.
--(Instances of their subtypes will conform to the collection's type but will also have additional attributes
--appropriate to their particular subtype. AsterixDB's open types make this possible.) In MongoDB, we would
--have these same collections. We just wouldn't bother to define any of their type info (and the system wouldn't
--do any checking); everything would be by convention/documentation, but the data would be exactly the same.
CREATE DATASET users(UserType) PRIMARY KEY userId AUTOGENERATED;
CREATE DATASET observations(ObservationType) PRIMARY KEY observationId AUTOGENERATED;
CREATE DATASET events(EventType) PRIMARY KEY eventId AUTOGENERATED;
CREATE DATASET datasources(DataSourceType) PRIMARY KEY dataSourceId AUTOGENERATED;
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)