You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@warble.apache.org by hu...@apache.org on 2018/06/25 03:12:08 UTC
[incubator-warble-server] branch master updated (d0bcf08 -> 2fbc641)
This is an automated email from the ASF dual-hosted git repository.
humbedooh pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-warble-server.git.
from d0bcf08 add gitignore
new 5f0c3f6 Initial checkout: a setup script!!
new 76b75a6 separate sessions and accounts
new 80def08 no longer need cookie var here
new ecfa34e initial skeleton of WSGI server
new 2fa6871 generate openapi yaml
new 1c06186 add main gitignore
new 2fbc641 start on a basic readme
The 7 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
api/.gitignore => .gitignore | 1 +
README.md | 12 +-
api/handler.py | 199 ++++++++++
api/pages/__init__.py | 46 +++
api/pages/account.py | 220 +++++++++++
api/pages/session.py | 185 ++++++++++
api/pages/widgets.py | 67 ++++
api/plugins/database.py | 103 ++++++
api/plugins/openapi.py | 259 +++++++++++++
api/plugins/session.py | 154 ++++++++
api/yaml/openapi.yaml | 403 +++++++++++++++++++++
api/yaml/openapi/combine.py | 145 ++++++++
.../components/schemas/ActionCompleted.yaml | 10 +
api/yaml/openapi/components/schemas/Empty.yaml | 11 +
api/yaml/openapi/components/schemas/Error.yaml | 16 +
.../openapi/components/schemas/Timeseries.yaml | 12 +
.../components/schemas/TimeseriesObject.yaml | 18 +
.../openapi/components/schemas/UserAccount.yaml | 20 +
.../components/schemas/UserCredentials.yaml | 15 +
api/yaml/openapi/components/schemas/WidgetApp.yaml | 37 ++
.../openapi/components/schemas/WidgetDesign.yaml | 10 +
api/yaml/openapi/components/schemas/WidgetRow.yaml | 10 +
.../components/schemas/defaultWidgetArgs.yaml | 79 ++++
.../components/securitySchemes/APIKeyAuth.yaml | 6 +
.../components/securitySchemes/cookieAuth.yaml | 6 +
setup/dbs.yaml | 34 ++
setup/setup.py | 122 +++++++
setup/warble.yaml.sample | 9 +
28 files changed, 2208 insertions(+), 1 deletion(-)
copy api/.gitignore => .gitignore (85%)
create mode 100644 api/handler.py
create mode 100644 api/pages/__init__.py
create mode 100644 api/pages/account.py
create mode 100644 api/pages/session.py
create mode 100644 api/pages/widgets.py
create mode 100644 api/plugins/database.py
create mode 100644 api/plugins/openapi.py
create mode 100644 api/plugins/session.py
create mode 100644 api/yaml/openapi.yaml
create mode 100644 api/yaml/openapi/combine.py
create mode 100644 api/yaml/openapi/components/schemas/ActionCompleted.yaml
create mode 100644 api/yaml/openapi/components/schemas/Empty.yaml
create mode 100644 api/yaml/openapi/components/schemas/Error.yaml
create mode 100644 api/yaml/openapi/components/schemas/Timeseries.yaml
create mode 100644 api/yaml/openapi/components/schemas/TimeseriesObject.yaml
create mode 100644 api/yaml/openapi/components/schemas/UserAccount.yaml
create mode 100644 api/yaml/openapi/components/schemas/UserCredentials.yaml
create mode 100644 api/yaml/openapi/components/schemas/WidgetApp.yaml
create mode 100644 api/yaml/openapi/components/schemas/WidgetDesign.yaml
create mode 100644 api/yaml/openapi/components/schemas/WidgetRow.yaml
create mode 100644 api/yaml/openapi/components/schemas/defaultWidgetArgs.yaml
create mode 100644 api/yaml/openapi/components/securitySchemes/APIKeyAuth.yaml
create mode 100644 api/yaml/openapi/components/securitySchemes/cookieAuth.yaml
create mode 100644 setup/dbs.yaml
create mode 100644 setup/setup.py
create mode 100644 setup/warble.yaml.sample
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@warble.apache.org
For additional commands, e-mail: commits-help@warble.apache.org
[incubator-warble-server] 04/07: initial skeleton of WSGI server
Posted by hu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
humbedooh pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-warble-server.git
commit ecfa34ee7c8ba2d3f1142796b6547d9d49e8ccaa
Author: Daniel Gruno <hu...@apache.org>
AuthorDate: Sun Jun 24 22:05:44 2018 -0500
initial skeleton of WSGI server
Technically supports both sqlite and ES, but let's stick with sqlite for
now
---
api/handler.py | 199 ++++++++++++++++
api/pages/__init__.py | 46 ++++
api/pages/account.py | 220 +++++++++++++++++
api/pages/session.py | 185 +++++++++++++++
api/pages/widgets.py | 67 ++++++
api/plugins/database.py | 103 ++++++++
api/plugins/openapi.py | 259 +++++++++++++++++++++
api/plugins/session.py | 154 ++++++++++++
api/yaml/openapi/combine.py | 145 ++++++++++++
.../components/schemas/ActionCompleted.yaml | 10 +
api/yaml/openapi/components/schemas/Empty.yaml | 11 +
api/yaml/openapi/components/schemas/Error.yaml | 16 ++
.../openapi/components/schemas/Timeseries.yaml | 12 +
.../components/schemas/TimeseriesObject.yaml | 18 ++
.../openapi/components/schemas/UserAccount.yaml | 20 ++
.../components/schemas/UserCredentials.yaml | 15 ++
api/yaml/openapi/components/schemas/WidgetApp.yaml | 37 +++
.../openapi/components/schemas/WidgetDesign.yaml | 10 +
api/yaml/openapi/components/schemas/WidgetRow.yaml | 10 +
.../components/schemas/defaultWidgetArgs.yaml | 79 +++++++
.../components/securitySchemes/APIKeyAuth.yaml | 6 +
.../components/securitySchemes/cookieAuth.yaml | 6 +
22 files changed, 1628 insertions(+)
diff --git a/api/handler.py b/api/handler.py
new file mode 100644
index 0000000..03b4d5c
--- /dev/null
+++ b/api/handler.py
@@ -0,0 +1,199 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+This is the main WSGI handler file for Apache Warble.
+It compiles a list of valid URLs from the 'pages' library folder,
+and if a URL matches it runs the specific submodule's run() function. It
+also handles CGI parsing and exceptions in the applications.
+"""
+
+
+# Main imports
+import cgi
+import re
+import sys
+import traceback
+import yaml
+import json
+import plugins.session
+import plugins.database
+import plugins.openapi
+
+WARBLE_VERSION = '0.1.0'
+
+# Compile valid API URLs from the pages library
+
+urls = []
+if __name__ != '__main__':
+ import pages
+ for page in pages.handlers:
+ urls.append((r"^(/api/%s)(/.+)?$" % page, pages.handlers[page].run))
+
+
+# Load Warble master configuration
+config = yaml.load(open("yaml/warble.yaml"))
+
+# Instantiate database connections
+DB = plugins.database.WarbleDatabase(config)
+
+# Load Open API specifications
+WarbleOpenAPI = plugins.openapi.OpenAPI("yaml/openapi.yaml")
+
+class WarbleHTTPError(Exception):
+ def __init__(self, code, message):
+ self.code = code
+ self.message = message
+
+
+class WarbleAPIWrapper:
+ """
+ Middleware wrapper for exceptions in the application
+ """
+ def __init__(self, path, func):
+ self.func = func
+ self.API = WarbleOpenAPI
+ self.path = path
+ self.exception = WarbleHTTPError
+
+ def __call__(self, environ, start_response, session):
+ """Run the function, return response OR return stacktrace"""
+ response = None
+ try:
+ # Read JSON client data if any
+ try:
+ request_size = int(environ.get('CONTENT_LENGTH', 0))
+ except (ValueError):
+ request_size = 0
+ requestBody = environ['wsgi.input'].read(request_size)
+ formdata = {}
+ if requestBody and len(requestBody) > 0:
+ try:
+ formdata = json.loads(requestBody.decode('utf-8'))
+ except json.JSONDecodeError as err:
+ start_response('400 Invalid request', [
+ ('Content-Type', 'application/json')])
+ yield json.dumps({
+ "code": 400,
+ "reason": "Invalid JSON: %s" % err,
+ 'server': "Apache Warble/%s" % WARBLE_VERSION
+ })
+ return
+
+ # Validate URL against OpenAPI specs
+ try:
+ self.API.validate(environ['REQUEST_METHOD'], self.path, formdata)
+ except plugins.openapi.OpenAPIException as err:
+ session.headers.append(('Content-Type', 'application/json'))
+ start_response('400 Invalid request',
+ session.headers)
+ yield json.dumps({
+ "code": 400,
+ "reason": err.message,
+ 'server': "Apache Warble/%s" % WARBLE_VERSION
+ })
+ return
+
+ # Call page with env, SR and form data
+ try:
+ response = self.func(self, environ, formdata, session)
+ if response:
+ for bucket in response:
+ yield bucket
+ except WarbleHTTPError as err:
+ errHeaders = {
+ 403: '403 Authentication failed',
+ 404: '404 Resource not found',
+ 500: '500 Internal Server Error',
+ 501: '501 Gateway error'
+ }
+ errHeader = errHeaders[err.code] if err.code in errHeaders else "400 Bad request"
+ session.headers.append(('Content-Type', 'application/json'))
+ start_response(errHeader, session.headers)
+ yield json.dumps({
+ "code": err.code,
+ "reason": err.message,
+ 'server': "Apache Warble/%s" % WARBLE_VERSION
+ }, indent = 4) + "\n"
+ return
+
+ except:
+ err_type, err_value, tb = sys.exc_info()
+ traceback_output = ['API traceback:']
+ traceback_output += traceback.format_tb(tb)
+ traceback_output.append('%s: %s' % (err_type.__name__, err_value))
+ # We don't know if response has been given yet, try giving one, fail gracefully.
+ try:
+ session.headers.append(('Content-Type', 'application/json'))
+ start_response('500 Internal Server Error',
+ session.headers)
+ except:
+ pass
+ yield json.dumps({
+ "code": "500",
+ "reason": '\n'.join(traceback_output),
+ 'server': "Apache Warble/%s" % WARBLE_VERSION
+ })
+
+
+def fourohfour(environ, start_response):
+ """A very simple 404 handler"""
+ start_response("404 Not Found", [
+ ('Content-Type', 'application/json')])
+ yield json.dumps({
+ "code": 404,
+ "reason": "API endpoint not found",
+ 'server': "Apache Warble/%s" % WARBLE_VERSION
+ }, indent = 4) + "\n"
+ return
+
+
+def application(environ, start_response):
+ """
+ This is the main handler. Every API call goes through here.
+ Checks against the pages library, and if submod found, runs
+ it and returns the output.
+ """
+ path = environ.get('PATH_INFO', '')
+ for regex, function in urls:
+ m = re.match(regex, path)
+ if m:
+ callback = WarbleAPIWrapper(path, function)
+ session = plugins.session.WarbleSession(DB, environ, config)
+ a = 0
+ for bucket in callback(environ, start_response, session):
+ if a == 0:
+ session.headers.append(bucket)
+ try:
+ start_response("200 Okay", session.headers)
+ except:
+ pass
+ a += 1
+ # WSGI prefers byte strings, so convert if regular py3 string
+ if isinstance(bucket, str):
+ yield bytes(bucket, encoding = 'utf-8')
+ elif isinstance(bucket, bytes):
+ yield bucket
+ return
+
+ for bucket in fourohfour(environ, start_response):
+ yield bytes(bucket, encoding = 'utf-8')
+
+
+
+if __name__ == '__main__':
+ WarbleOpenAPI.toHTML()
diff --git a/api/pages/__init__.py b/api/pages/__init__.py
new file mode 100644
index 0000000..e412905
--- /dev/null
+++ b/api/pages/__init__.py
@@ -0,0 +1,46 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+"""
+Warble API scripts library:
+
+ oauth: oauth manager
+
+"""
+
+import importlib
+import os
+# Define all the submodules we have
+
+rootpath = os.path.dirname(__file__)
+print("Reading pages from %s" % rootpath)
+
+# Import each submodule into a hash called 'handlers'
+handlers = {}
+
+def loadPage(path):
+ for el in os.listdir(path):
+ filepath = os.path.join(path, el)
+ if el.find("__") == -1:
+ if os.path.isdir(filepath):
+ loadPage(filepath)
+ else:
+ p = filepath.replace(rootpath, "")[1:].replace('/', '.')[:-3]
+ xp = p.replace('.', '/')
+ print("Loading endpoint pages.%s as %s" % (p, xp))
+ handlers[xp] = importlib.import_module("pages.%s" % p)
+
+loadPage(rootpath)
diff --git a/api/pages/account.py b/api/pages/account.py
new file mode 100644
index 0000000..4857773
--- /dev/null
+++ b/api/pages/account.py
@@ -0,0 +1,220 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+########################################################################
+# OPENAPI-URI: /api/account
+########################################################################
+# delete:
+# requestBody:
+# content:
+# application/json:
+# schema:
+# $ref: '#/components/schemas/UserName'
+# description: User ID
+# required: true
+# responses:
+# '200':
+# content:
+# application/json:
+# schema:
+# $ref: '#/components/schemas/ActionCompleted'
+# description: 200 response
+# default:
+# content:
+# application/json:
+# schema:
+# $ref: '#/components/schemas/Error'
+# description: unexpected error
+# security:
+# - cookieAuth: []
+# summary: Delete an account
+# patch:
+# requestBody:
+# content:
+# application/json:
+# schema:
+# $ref: '#/components/schemas/UserAccountEdit'
+# description: User credentials
+# required: true
+# responses:
+# '200':
+# content:
+# application/json:
+# schema:
+# $ref: '#/components/schemas/ActionCompleted'
+# description: 200 response
+# default:
+# content:
+# application/json:
+# schema:
+# $ref: '#/components/schemas/Error'
+# description: unexpected error
+# summary: Edit an account
+# put:
+# requestBody:
+# content:
+# application/json:
+# schema:
+# $ref: '#/components/schemas/UserAccount'
+# description: User credentials
+# required: true
+# responses:
+# '200':
+# content:
+# application/json:
+# schema:
+# $ref: '#/components/schemas/ActionCompleted'
+# description: 200 response
+# default:
+# content:
+# application/json:
+# schema:
+# $ref: '#/components/schemas/Error'
+# description: unexpected error
+# summary: Create a new account
+#
+########################################################################
+
+
+
+
+
+"""
+This is the user account handler for Kibble.
+adds, removes and edits accounts.
+"""
+
+import json
+import re
+import time
+import bcrypt
+import hashlib
+import smtplib
+import email.message
+
+
+def sendCode(session, addr, code):
+ msg = email.message.EmailMessage()
+ msg['To'] = addr
+ msg['From'] = session.config['mail']['sender']
+ msg['Subject'] = "Please verify your account"
+ msg.set_content("""\
+Hi there!
+Please verify your account by visiting:
+%s/api/verify/%s/%s
+
+With regards,
+Apache Kibble.
+""" % (session.url, addr, code)
+ )
+ s = smtplib.SMTP("%s:%s" % (session.config['mail']['mailhost'], session.config['mail']['mailport']))
+ s.send_message(msg)
+ s.quit()
+
+def run(API, environ, indata, session):
+
+ method = environ['REQUEST_METHOD']
+
+ # Add a new account??
+ if method == "PUT":
+ u = indata['email']
+ p = indata['password']
+ d = indata['displayname']
+
+ # Are new accounts allowed? (admin can always make accounts, of course)
+ if not session.config['accounts'].get('allowSignup', False):
+ if not (session.user and session.user['level'] == 'admin'):
+ raise API.exception(403, "New account requests have been administratively disabled.")
+
+ # Check if we already have that username in use
+ if session.DB.ES.exists(index=session.DB.dbname, doc_type='useraccount', id = u):
+ raise API.exception(403, "Username already in use")
+
+ # We require a username, displayName password of at least 3 chars each
+ if len(p) < 3 or len(u) < 3 or len(d) < 3:
+ raise API.exception(400, "Username, display-name and password must each be at elast 3 characters long.")
+
+ # We loosely check that the email is an email
+ if not re.match(r"^\S+@\S+\.\S+$", u):
+ raise API.exception(400, "Invalid email address presented.")
+
+ # Okay, let's make an account...I guess
+ salt = bcrypt.gensalt()
+ pwd = bcrypt.hashpw(p.encode('utf-8'), salt).decode('ascii')
+
+ # Verification code, if needed
+ vsalt = bcrypt.gensalt()
+ vcode = hashlib.sha1(vsalt).hexdigest()
+
+ # Auto-verify unless verification is enabled.
+ # This is so previously unverified accounts don'thave to verify
+ # if we later turn verification on.
+ verified = True
+ if session.config['accounts'].get('verify'):
+ verified = False
+ sendCode(session, u, vcode) # Send verification email
+ # If verification email fails, skip account creation.
+
+ doc = {
+ 'email': u, # Username (email)
+ 'password': pwd, # Hashed password
+ 'displayName': d, # Display Name
+ 'organisations': [], # Orgs user belongs to (default is none)
+ 'ownerships': [], # Orgs user owns (default is none)
+ 'defaultOrganisation': None, # Default org for user
+ 'verified': verified, # Account verified via email?
+ 'vcode': vcode, # Verification code
+ 'userlevel': "user" # User level (user/admin)
+ }
+
+
+ # If we have auto-invite on, check if there are orgs to invite to
+ if 'autoInvite' in session.config['accounts']:
+ dom = u.split('@')[-1].lower()
+ for ai in session.config['accounts']['autoInvite']:
+ if ai['domain'] == dom:
+ doc['organisations'].append(ai['organisation'])
+
+ session.DB.ES.index(index=session.DB.dbname, doc_type='useraccount', id = u, body = doc)
+ yield json.dumps({"message": "Account created!", "verified": verified})
+ return
+
+ # We need to be logged in for the rest of this!
+ if not session.user:
+ raise API.exception(403, "You must be logged in to use this API endpoint! %s")
+
+
+ # Patch (edit) an account
+ if method == "PATCH":
+ userid = session.user['email']
+ if indata.get('email') and session.user['userlevel'] == "admin":
+ userid = indata.get('email')
+ doc = session.DB.ES.get(index=session.DB.dbname, doc_type='useraccount', id = userid)
+ udoc = doc['_source']
+ if indata.get('defaultOrganisation'):
+ # Make sure user is a member or admin here..
+ if session.user['userlevel'] == "admin" or indata.get('defaultOrganisation') in udoc['organisations']:
+ udoc['defaultOrganisation'] = indata.get('defaultOrganisation')
+ # Changing pasword?
+ if indata.get('password'):
+ p = indata.get('password')
+ salt = bcrypt.gensalt()
+ pwd = bcrypt.hashpw(p.encode('utf-8'), salt).decode('ascii')
+ # Update user doc
+ session.DB.ES.index(index=session.DB.dbname, doc_type='useraccount', id = userid, body = udoc)
+ yield json.dumps({"message": "Account updated!"})
+ return
+
\ No newline at end of file
diff --git a/api/pages/session.py b/api/pages/session.py
new file mode 100644
index 0000000..87f8938
--- /dev/null
+++ b/api/pages/session.py
@@ -0,0 +1,185 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+########################################################################
+# OPENAPI-URI: /api/session
+########################################################################
+# delete:
+# requestBody:
+# content:
+# application/json:
+# schema:
+# $ref: '#/components/schemas/Empty'
+# description: Nada
+# required: true
+# responses:
+# '200':
+# content:
+# application/json:
+# schema:
+# $ref: '#/components/schemas/ActionCompleted'
+# description: Logout successful
+# default:
+# content:
+# application/json:
+# schema:
+# $ref: '#/components/schemas/Error'
+# description: unexpected error
+# security:
+# - cookieAuth: []
+# summary: Log out (remove session)
+# get:
+# responses:
+# '200':
+# content:
+# application/json:
+# schema:
+# $ref: '#/components/schemas/UserData'
+# description: 200 response
+# default:
+# content:
+# application/json:
+# schema:
+# $ref: '#/components/schemas/Error'
+# description: unexpected error
+# security:
+# - cookieAuth: []
+# summary: Display your login details
+# put:
+# requestBody:
+# content:
+# application/json:
+# schema:
+# $ref: '#/components/schemas/UserCredentials'
+# description: User credentials
+# required: true
+# responses:
+# '200':
+# content:
+# application/json:
+# schema:
+# $ref: '#/components/schemas/ActionCompleted'
+# description: Login successful
+# headers:
+# Set-Cookie:
+# schema:
+# example: 77488a26-23c2-4e29-94a1-6a0738f6a3ff
+# type: string
+# default:
+# content:
+# application/json:
+# schema:
+# $ref: '#/components/schemas/Error'
+# description: unexpected error
+# summary: Log in
+#
+########################################################################
+
+
+
+
+
+"""
+This is the user session handler for Warble
+"""
+
+import json
+import re
+import time
+import bcrypt
+import hashlib
+
+def run(API, environ, indata, session):
+
+ method = environ['REQUEST_METHOD']
+
+ # Logging in?
+ if method == "PUT":
+ u = indata['username']
+ p = indata['password']
+ if session.DB.dbtype == 'sqlite':
+ session_conn = session.DB.sqlite.open('sessions.db')
+ account_conn = session.DB.sqlite.open('accounts.db')
+ sc = session_conn.cursor()
+ ac = account_conn.cursor()
+ ac.execute("SELECT * FROM `accounts` WHERE `userid` = ? LIMIT 1", (u,))
+ sdoc = ac.fetchone()
+ if sdoc:
+ mypass = sdoc['password']
+ theirpass = bcrypt.hashpw(p.encode('utf-8'), mypass.encode('utf-8')).decode('ascii')
+ if mypass == theirpass:
+ sc.execute("INSERT INTO `sessions` (`userid`, `cookie`, `timestamp`) VALUES (?, ?, ?)",
+ (u, session.cookie, int(time.time())))
+ session_conn.commit()
+ session_conn.close()
+ account_conn.close()
+ yield json.dumps({"message": "Logged in OK!"}, indent = 2)
+ return
+ # Fall back to a 403 if username and password did not match
+ raise API.exception(403, "Wrong username or password supplied!")
+
+ elif session.DB.dbtype == 'elasticsearch':
+ if session.DB.ES.exists(index=session.DB.dbname, doc_type='useraccount', id = u):
+ doc = session.DB.ES.get(index=session.DB.dbname, doc_type='useraccount', id = u)
+ hp = doc['_source']['password']
+ if bcrypt.hashpw(p.encode('utf-8'), hp.encode('utf-8')).decode('ascii') == hp:
+ # If verification is enabled, make sure account is verified
+ if session.config['accounts'].get('verify'):
+ if doc['_source']['verified'] == False:
+ raise API.exception(403, "Your account needs to be verified first. Check your inbox!")
+ sessionDoc = {
+ 'cid': u,
+ 'id': session.cookie,
+ 'timestamp': int(time.time())
+ }
+ session.DB.ES.index(index=session.DB.dbname, doc_type='uisession', id = session.cookie, body = sessionDoc)
+ yield json.dumps({"message": "Logged in OK!"})
+ return
+
+ # Fall back to a 403 if username and password did not match
+ raise API.exception(403, "Wrong username or password supplied!")
+
+
+ # We need to be logged in for the rest of this!
+ if not session.user:
+ raise API.exception(403, "You must be logged in to use this API endpoint!")
+
+ # Delete a session (log out)
+ if method == "DELETE":
+ if self.DB.dbtype == 'sqlite':
+ c = self.DB.sqlite.open('sessions.db')
+ cur = c.cursor()
+ cur.execute("DELETE FROM `sessions` WHERE `cookie` = ? LIMIT 1", (session.cookie,))
+ c.commit()
+ c.close()
+ elif self.DB.dbtype == 'elasticsearch':
+ session.DB.ES.delete(index=session.DB.dbname, doc_type='uisession', id = session.cookie)
+ session.newCookie()
+ yield json.dumps({"message": "Logged out, bye bye!"})
+
+ # Display the user data for this session
+ if method == "GET":
+
+ JSON_OUT = {
+ 'userid': session.user['userid'],
+ 'userlevel': session.user['userlevel']
+ }
+ yield json.dumps(JSON_OUT, indent = 2)
+ return
+
+ # Finally, if we hit a method we don't know, balk!
+ yield API.exception(400, "I don't know this request method!!")
+
diff --git a/api/pages/widgets.py b/api/pages/widgets.py
new file mode 100644
index 0000000..d25d0c7
--- /dev/null
+++ b/api/pages/widgets.py
@@ -0,0 +1,67 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+########################################################################
+# OPENAPI-URI: /api/widgets/{pageid}
+########################################################################
+# get:
+# summary: Shows the widget layout for a specific page
+# security:
+# - cookieAuth: []
+# parameters:
+# - name: pageid
+# in: path
+# description: Page ID to fetch design for
+# required: true
+# schema:
+# type: string
+# responses:
+# '200':
+# description: 200 Response
+# content:
+# application/json:
+# schema:
+# $ref: '#/components/schemas/WidgetDesign'
+# default:
+# description: unexpected error
+# content:
+# application/json:
+# schema:
+# $ref: '#/components/schemas/Error'
+########################################################################
+"""
+This is the widget design handler for Warble
+"""
+
+import yaml
+import json
+
+def run(API, environ, indata, session):
+
+ if not session.user:
+ raise API.exception(403, "You must be logged in to use this API endpoint!")
+
+ widgets = yaml.load(open("yaml/widgets.yaml"))
+
+ page = indata['pageid']
+ if not page or page == '0':
+ page = widgets.get('defaultWidget', 'repos')
+ if page in widgets['widgets']:
+ yield json.dumps(widgets['widgets'][page])
+ else:
+ raise API.exception(404, "Widget design not found!")
+
+
diff --git a/api/plugins/database.py b/api/plugins/database.py
new file mode 100644
index 0000000..53eb2cd
--- /dev/null
+++ b/api/plugins/database.py
@@ -0,0 +1,103 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+This is the ES/sqlite library for Apache Warble.
+It stores the elasticsearch handler and config options.
+"""
+
+
+# Main imports
+import os
+import sqlite3
+
+class WarbleESWrapper(object):
+ """
+ Class for rewriting old-style queries to the new ones,
+ where doc_type is an integral part of the DB name
+ """
+ def __init__(self, ES):
+ self.ES = ES
+
+ def get(self, index, doc_type, id):
+ return self.ES.get(index = index+'_'+doc_type, doc_type = '_doc', id = id)
+ def exists(self, index, doc_type, id):
+ return self.ES.exists(index = index+'_'+doc_type, doc_type = '_doc', id = id)
+ def delete(self, index, doc_type, id):
+ return self.ES.delete(index = index+'_'+doc_type, doc_type = '_doc', id = id)
+ def index(self, index, doc_type, id, body):
+ return self.ES.index(index = index+'_'+doc_type, doc_type = '_doc', id = id, body = body)
+ def update(self, index, doc_type, id, body):
+ return self.ES.update(index = index+'_'+doc_type, doc_type = '_doc', id = id, body = body)
+ def search(self, index, doc_type, size = 100, _source_include = None, body = None):
+ return self.ES.search(
+ index = index+'_'+doc_type,
+ doc_type = '_doc',
+ size = size,
+ _source_include = _source_include,
+ body = body
+ )
+ def count(self, index, doc_type, body = None):
+ return self.ES.count(
+ index = index+'_'+doc_type,
+ doc_type = '_doc',
+ body = body
+ )
+
+class WarbleSqlite(object):
+
+ def __init__(self, path):
+ self.path = path
+
+ def open(self, file):
+ c = sqlite3.connect(os.path.join(self.path, file))
+ c.row_factory = sqlite3.Row
+ return c
+
+
+class WarbleDatabase(object):
+ def __init__(self, config):
+ self.config = config
+
+ # sqlite driver?
+ if self.config['database']['driver'] == 'sqlite':
+ self.dbtype = 'sqlite'
+ self.sqlite = WarbleSqlite(self.config['database']['path'])
+
+ # ES driver?
+ if self.config['database']['driver'] == 'elasticsearch':
+ import elasticsearch
+ self.dbtype = 'elasticsearch'
+ self.dbname = config['elasticsearch']['dbname']
+ self.ES = elasticsearch.Elasticsearch([{
+ 'host': config['elasticsearch']['host'],
+ 'port': int(config['elasticsearch']['port']),
+ 'use_ssl': config['elasticsearch']['ssl'],
+ 'verify_certs': False,
+ 'url_prefix': config['elasticsearch']['uri'] if 'uri' in config['elasticsearch'] else '',
+ 'http_auth': config['elasticsearch']['auth'] if 'auth' in config['elasticsearch'] else None
+ }],
+ max_retries=5,
+ retry_on_timeout=True
+ )
+
+ # IMPORTANT BIT: Figure out if this is ES 6.x or newer.
+ # If so, we're using the new ES DB mappings, and need to adjust ALL
+ # ES calls to match this.
+ es6 = True if int(self.ES.info()['version']['number'].split('.')[0]) >= 6 else False
+ if es6:
+ self.ES = WarbleESWrapper(self.ES)
diff --git a/api/plugins/openapi.py b/api/plugins/openapi.py
new file mode 100644
index 0000000..71a7117
--- /dev/null
+++ b/api/plugins/openapi.py
@@ -0,0 +1,259 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+This is the OpenAPI validator library.
+Validates input using the OpenAPI specification version 3 from
+https://github.com/OAI/OpenAPI-Specification (a simplified version, ahem)
+"""
+
+import yaml
+import json
+import functools
+import operator
+import re
+
+class OpenAPIException(Exception):
+ def __init__(self, message):
+ self.message = message
+
+# Python type names to JSON type names
+py2JSON = {
+ 'int': 'integer',
+ 'float': 'float',
+ 'str': 'string',
+ 'list': 'array',
+ 'dict': 'object',
+ 'bool': 'boolean'
+}
+
+mcolors = {
+ 'PUT': '#fca130',
+ 'DELETE': '#f93e3e',
+ 'GET': '#61affe',
+ 'POST': '#49cc5c',
+ 'PATCH': '#d5a37e'
+}
+
+class OpenAPI():
+ def __init__(self, APIFile):
+ """ Instantiates an OpenAPI validator given a YAML specification"""
+ if APIFile.endswith(".json") or APIFile.endswith(".js"):
+ self.API = json.load(open(APIFile))
+ else:
+ self.API = yaml.load(open(APIFile))
+
+ def validateType(self, field, value, ftype):
+ """ Validate a single field value against an expected type """
+
+ # Get type of value, convert to JSON name of type.
+ pyType = type(value).__name__
+ jsonType = py2JSON[pyType] if pyType in py2JSON else pyType
+
+ # Check if type matches
+ if ftype != jsonType:
+ raise OpenAPIException("OpenAPI mismatch: Field '%s' was expected to be %s, but was really %s!" % (field, ftype, jsonType))
+
+ def validateSchema(self, pdef, formdata, schema = None):
+ """ Validate (sub)parameters against OpenAPI specs """
+
+ # allOf: list of schemas to validate against
+ if 'allOf' in pdef:
+ for subdef in pdef['allOf']:
+ self.validateSchema(subdef, formdata)
+
+ where = "JSON body"
+ # Symbolic link??
+ if 'schema' in pdef:
+ schema = pdef['schema']['$ref']
+ if '$ref' in pdef:
+ schema = pdef['$ref']
+ if schema:
+ # #/foo/bar/baz --> dict['foo']['bar']['baz']
+ pdef = functools.reduce(operator.getitem, schema.split('/')[1:], self.API)
+ where = "item matching schema %s" % schema
+
+ # Check that all required fields are present
+ if 'required' in pdef:
+ for field in pdef['required']:
+ if not field in formdata:
+ raise OpenAPIException("OpenAPI mismatch: Missing input field '%s' in %s!" % (field, where))
+
+ # Now check for valid format of input data
+ for field in formdata:
+ if 'properties' not in pdef or field not in pdef['properties'] :
+ raise OpenAPIException("Unknown input field '%s' in %s!" % (field, where))
+ if 'type' not in pdef['properties'][field]:
+ raise OpenAPIException("OpenAPI mismatch: Field '%s' was found in api.yaml, but no format was specified in specs!" % field)
+ ftype = pdef['properties'][field]['type']
+ self.validateType(field, formdata[field], ftype)
+
+ # Validate sub-arrays
+ if ftype == 'array' and 'items' in pdef['properties'][field]:
+ for item in formdata[field]:
+ if '$ref' in pdef['properties'][field]['items']:
+ self.validateSchema(pdef['properties'][field]['items'], item)
+ else:
+ self.validateType(field, formdata[field], pdef['properties'][field]['items']['type'])
+
+ # Validate sub-hashes
+ if ftype == 'hash' and 'schema' in pdef['properties'][field]:
+ self.validateSchema(pdef['properties'][field], formdata[field])
+ def validateParameters(self, defs, formdata):
+ #
+ pass
+
+ def validate(self, method = "GET", path = "/foo", formdata = None):
+ """ Validate the request method and input data against the OpenAPI specification """
+
+ # Make sure we're not dealing with a dynamic URL.
+ # If we find /foo/{key}, we fold that into the form data
+ # and process as if it's a json input field for now.
+ if not self.API['paths'].get(path):
+ for xpath in self.API['paths']:
+ pathRE = re.sub(r"\{(.+?)\}", r"(?P<\1>[^/]+)", xpath)
+ m = re.match(pathRE, path)
+ if m:
+ for k,v in m.groupdict().items():
+ formdata[k] = v
+ path = xpath
+ break
+
+ if self.API['paths'].get(path):
+ defs = self.API['paths'].get(path)
+ method = method.lower()
+ if method in defs:
+ mdefs = defs[method]
+ if formdata and 'parameters' in mdefs:
+ self.validateParameters(mdefs['parameters'], formdata)
+ elif formdata and 'requestBody' not in mdefs:
+ raise OpenAPIException("OpenAPI mismatch: JSON data is now allowed for this request type")
+ elif formdata and 'requestBody' in mdefs and 'content' in mdefs['requestBody']:
+
+ # SHORTCUT: We only care about JSON input for Warble! Disregard other types
+ if not 'application/json' in mdefs['requestBody']['content']:
+ raise OpenAPIException ("OpenAPI mismatch: API endpoint accepts input, but no application/json definitions found in api.yaml!")
+ jdefs = mdefs['requestBody']['content']['application/json']
+
+ # Check that required params are here
+ self.validateSchema(jdefs, formdata)
+
+ else:
+ raise OpenAPIException ("OpenAPI mismatch: Method %s is not registered for this API" % method)
+ else:
+ raise OpenAPIException("OpenAPI mismatch: Unknown API path '%s'!" % path)
+
+ def dumpExamples(self, pdef, array = False):
+ schema = None
+ if 'schema' in pdef:
+ if 'type' in pdef['schema'] and pdef['schema']['type'] == 'array':
+ array = True
+ schema = pdef['schema']['items']['$ref']
+ else:
+ schema = pdef['schema']['$ref']
+ if '$ref' in pdef:
+ schema = pdef['$ref']
+ if schema:
+ # #/foo/bar/baz --> dict['foo']['bar']['baz']
+ pdef = functools.reduce(operator.getitem, schema.split('/')[1:], self.API)
+ js = {}
+ desc = {}
+ if 'properties' in pdef:
+ for k, v in pdef['properties'].items():
+ if 'description' in v:
+ desc[k] = [v['type'], v['description']]
+ if 'example' in v:
+ js[k] = v['example']
+ elif 'items' in v:
+ if v['type'] == 'array':
+ js[k], foo = self.dumpExamples(v['items'], True)
+ else:
+ js[k], foo = self.dumpExamples(v['items'])
+ return [js if not array else [js], desc]
+
+ def toHTML(self):
+ """ Blurps out the specs in a pretty HTML blob """
+ print("""
+<!DOCTYPE html>
+<html lang="en">
+<head>
+</head>
+<body>
+""")
+ li = "<h3>Overview:</h3><ul style='font-size: 12px; font-family: Open Sans, sans-serif;'>"
+ for path, spec in sorted(self.API['paths'].items()):
+ for method, mspec in sorted(spec.items()):
+ method = method.upper()
+ summary = mspec.get('summary', 'No summary available')
+ linkname = "%s%s" % (method.lower(), path.replace('/', '-'))
+ li += "<li><a href='#%s'>%s %s</a>: %s</li>\n" % (linkname, method, path, summary)
+ li += "</ul>"
+ print(li)
+ for path, spec in sorted(self.API['paths'].items()):
+ for method, mspec in sorted(spec.items()):
+ method = method.upper()
+ summary = mspec.get('summary', 'No summary available')
+ resp = ""
+ inp = ""
+ inpvars = ""
+ linkname = "%s%s" % (method.lower(), path.replace('/', '-'))
+ if 'responses' in mspec:
+ for code, cresp in sorted(mspec['responses'].items()):
+ for ctype, pdef in cresp['content'].items():
+ xjs, desc = self.dumpExamples(pdef)
+ js = json.dumps(xjs, indent = 4)
+ resp += "<div style='float: left; width: 90%%;'><pre style='width: 600px;'><b>%s</b>:\n%s</pre>\n</div>\n" % (code, js)
+
+ if 'requestBody' in mspec:
+ for ctype, pdef in mspec['requestBody']['content'].items():
+ xjs, desc = self.dumpExamples(pdef)
+ if desc:
+ for k, v in desc.items():
+ inpvars += "<kbd><b>%s:</b></kbd> (%s) <span style='font-size: 12px; font-family: Open Sans, sans-serif;'>%s</span><br/>\n" % (k, v[0], v[1])
+ js = json.dumps(xjs, indent = 4)
+ inp += "<div style='float: left; width: 90%%;'><h4>Input examples:</h4><blockquote><pre style='width: 600px;'><b>%s</b>:\n%s</pre></blockquote>\n</div>" % (ctype, js)
+
+ if inpvars:
+ inpvars = "<div style='float: left; width: 90%%;'><blockquote><pre style='width: 600px;'>%s</pre>\n</blockquote></div>" % inpvars
+
+
+ print("""
+ <div id="%s" style="margin: 20px; display: flex; box-sizing: border-box; width: 900px; border-radius: 6px; border: 1px solid %s; font-family: sans-serif; background: %s30;">
+ <div style="min-height: 32px;">
+ <!-- method -->
+
+ <div style="float: left; align-items: center; margin: 4px; border-radius: 5px; text-align: center; padding-top: 4px; height: 20px; width: 100px; color: #FFF; font-weight: bold; background: %s;">%s</div>
+
+ <!-- path and summary -->
+ <span style="display: flex; padding-top: 6px;"><kbd><strong>%s</strong></kbd></span>
+ <div style="box-sizing: border-box; flex: 1; font-size: 13px; font-family: Open Sans, sans-serif; float: left; padding-top: 6px; margin-left: 20px;">
+ %s</div>
+ <div style="float: left; width: 90%%;display: %s; ">
+ <h4>JSON parameters:</h4>
+ %s
+ <br/>
+ %s
+ </div>
+ <div style="float: left; width: 90%%; ">
+ <h4>Response examples:</h4>
+ <blockquote>%s</blockquote>
+ </div>
+ </div>
+ </div>
+ """ % (linkname, mcolors[method], mcolors[method], mcolors[method], method, path, summary, "block" if inp else "none", inpvars, inp, resp))
+ #print("%s %s: %s" % (method.upper(), path, mspec['summary']))
+ print("</body></html>")
diff --git a/api/plugins/session.py b/api/plugins/session.py
new file mode 100644
index 0000000..ba20291
--- /dev/null
+++ b/api/plugins/session.py
@@ -0,0 +1,154 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+This is the session library for Apache Warble.
+It handles setting/getting cookies and user prefs
+"""
+
+
+# Main imports
+import cgi
+import re
+import sys
+import traceback
+import http.cookies
+import uuid
+import time
+
+class WarbleSession(object):
+
+ def logout(self):
+ """Log out user and wipe cookie"""
+ if self.user and self.cookie:
+ cookies = http.cookies.SimpleCookie()
+ cookies['warble_session'] = "null"
+ self.headers.append(('Set-Cookie', cookies['warble_session'].OutputString()))
+ try:
+ if self.DB.dbtype == 'sqlite':
+ c = self.DB.sqlite.open('sessions.db')
+ cur = c.cursor()
+ cur.execute("DELETE FROM `sessions` WHERE `cookie` = ? LIMIT 1", (self.cookie,))
+ c.commit()
+ c.close()
+ elif self.DB.dbtype == 'elasticsearch':
+ self.DB.ES.delete(index=self.DB.dbname, doc_type='uisession', id = self.cookie)
+
+ self.cookie = None
+ self.user = None
+ except:
+ pass
+ def newCookie(self):
+ cookie = uuid.uuid4()
+ cookies = http.cookies.SimpleCookie()
+ cookies['warble_session'] = cookie
+ cookies['warble_session']['expires'] = 86400 * 365 # Expire one year from now
+ self.headers.append(('Set-Cookie', cookies['warble_session'].OutputString()))
+
+ def __init__(self, DB, environ, config):
+ """
+ Loads the current user session or initiates a new session if
+ none was found.
+ """
+ self.config = config
+ self.user = None
+ self.DB = DB
+ self.headers = [('Content-Type', 'application/json')]
+ self.cookie = None
+
+ # Construct the URL we're visiting
+ self.url = "%s://%s" % (environ['wsgi.url_scheme'], environ.get('HTTP_HOST', environ.get('SERVER_NAME')))
+ self.url += environ.get('SCRIPT_NAME', '/')
+
+ # Get Warble cookie
+ cookie = None
+ cookies = None
+
+ if 'HTTP_COOKIE' in environ:
+ cookies = http.cookies.SimpleCookie(environ['HTTP_COOKIE'])
+ if cookies and 'warble_session' in cookies:
+ cookie = cookies['warble_session'].value
+ try:
+ if re.match(r"^[-a-f0-9]+$", cookie): # Validate cookie, must follow UUID4 specs
+ doc = None
+ if self.DB.dbtype == 'sqlite':
+ session_conn = self.DB.sqlite.open('sessions.db')
+ account_conn = self.DB.sqlite.open('accounts.db')
+ sc = session_conn.cursor()
+ ac = account_conn.cursor()
+ sc.execute("SELECT * FROM `sessions` WHERE `cookie` = ?", (cookie,))
+ sdoc = sc.fetchone()
+ if sdoc:
+ userid = sdoc['userid']
+ if userid:
+ sc.execute("SELECT * FROM `accounts` WHERE `userid` = ?", (userid,))
+ doc = sc.fetchone()
+ if doc:
+ # Make sure this cookie has been used in the past 7 days, else nullify it.
+ # Further more, run an update of the session if >1 hour ago since last update.
+ age = time.time() - sdoc['timestamp']
+ if age > (7*86400):
+ sc.execute("DELETE FROM `sessions` WHERE `cookie` = ? LIMIT 1", (self.cookie,))
+ sdoc = None # Wipe it!
+ doc = None
+ elif age > 3600:
+ sdoc['timestamp'] = int(time.time()) # Update timestamp in session DB
+ sc.execute("UPDATE `sessions` SET `timestamp` = ? WHERE `cookie` = ? LIMIT 1", (sdoc['timestamp'], cookie,))
+ if doc:
+ self.user = doc
+ self.user['userlevel'] = 'superuser'if doc['superuser'] else 'normal'
+ session_conn.commit()
+ session_conn.close()
+ account_conn.close()
+
+ if self.DB.dbtype == 'elasticsearch':
+ sdoc = self.DB.ES.get(index=self.DB.dbname, doc_type='uisession', id = cookie)
+ if sdoc and 'cid' in sdoc['_source']:
+ doc = self.DB.ES.get(index=self.DB.dbname, doc_type='useraccount', id = sdoc['_source']['cid'])
+ if doc and '_source' in doc:
+ # Make sure this cookie has been used in the past 7 days, else nullify it.
+ # Further more, run an update of the session if >1 hour ago since last update.
+ age = time.time() - sdoc['_source']['timestamp']
+ if age > (7*86400):
+ self.DB.ES.delete(index=self.DB.dbname, doc_type='uisession', id = cookie)
+ sdoc['_source'] = None # Wipe it!
+ doc = None
+ elif age > 3600:
+ sdoc['_source']['timestamp'] = int(time.time()) # Update timestamp in session DB
+ self.DB.ES.update(index=self.DB.dbname, doc_type='uisession', id = cookie, body = {'doc':sdoc['_source']})
+ if doc:
+ self.user = doc['_source']
+ else:
+ cookie = None
+ except Exception as err:
+ print(err)
+ # Non-human (node/agent) API Key auth
+ elif 'HTTP_APIKEY' in environ:
+ cookie = environ['HTTP_APIKEY']
+ if re.match(r"^[-a-f0-9]+$", cookie): # Validate cookie, must follow UUID4 specs
+ registry_conn = self.DB.sqlite.open('nodes.db')
+ rc = registry_conn.cursor()
+ rc.execute("SELECT * FROM `registry` WHERE `apikey` = ? LIMIT 1", (cookie,))
+ ndoc = rc.fetchone()
+ if ndoc:
+ self.user = {k:ndoc[k] for k in ndoc.keys()}
+ self.user['human'] = False
+ self.user['userid'] = 'node:%s' % ndoc['id']
+ self.user['userlevel'] = 'robbit'
+ if not cookie:
+ self.newCookie()
+ self.cookie = cookie
diff --git a/api/yaml/openapi/combine.py b/api/yaml/openapi/combine.py
new file mode 100644
index 0000000..30f9806
--- /dev/null
+++ b/api/yaml/openapi/combine.py
@@ -0,0 +1,145 @@
+#!/usr/bin/env python3
+
+import yaml
+import os
+import sys
+import re
+
+license = """#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""
+
+baseyaml = """
+# THIS IS PULLED FROM SCRIPTS AND AUTOGENERATED!
+# Please use openapi/combine.py to regenerate!
+openapi: 3.0.0
+info:
+ version: 1.0.0
+ description: This is the API specifications for interacting with the Warble Server.
+ title: Apache Warble API
+ license:
+ name: Apache 2.0
+ url: 'http://www.apache.org/licenses/LICENSE-2.0.html'
+"""
+
+bpath = os.path.dirname(os.path.abspath(__file__))
+
+
+def deconstruct():
+ yml = yaml.load(open(bpath + "/../openapi.yaml"))
+ noDefs = 0
+ print("Dumping paths into pages...")
+ for endpoint, defs in yml['paths'].items():
+ noDefs += 1
+ xendpoint = endpoint.replace("/api/", "")
+ ypath = os.path.abspath("%s/../../pages/%s.py" % (bpath, xendpoint))
+ print(ypath)
+ if os.path.isfile(ypath):
+ print("Editing %s" % ypath)
+ contents = open(ypath, "r").read()
+ contents = re.sub(r"^([#\n](?!\s*\"\"\")[^\r\n]*\n?)+", "", contents, re.MULTILINE)
+ odefs = yaml.dump(defs, default_flow_style=False)
+ odefs = "\n".join(["# %s" % line for line in odefs.split("\n")])
+ with open(ypath, "w") as f:
+ f.write(license)
+ f.write("########################################################################\n")
+ f.write("# OPENAPI-URI: %s\n" % endpoint)
+ f.write("########################################################################\n")
+ f.write(odefs)
+ f.write("\n########################################################################\n")
+ f.write("\n\n")
+ f.write(contents)
+ f.close()
+
+ print("Dumping security components...")
+ for basetype, bdefs in yml['components'].items():
+ for schema, defs in bdefs.items():
+ noDefs += 1
+ ypath = "%s/components/%s/%s.yaml" % (bpath, basetype, schema)
+ ydir = os.path.dirname(ypath)
+ if not os.path.isdir(ydir):
+ print("Making directory %s" % ydir)
+ os.makedirs(ydir, exist_ok = True)
+ with open(ypath, "w") as f:
+ f.write("########################################################################\n")
+ f.write("# %-68s #\n" % defs.get('summary', schema))
+ f.write("########################################################################\n")
+ f.write(yaml.dump(defs, default_flow_style=False))
+ f.close()
+ print("Dumped %u definitions." % noDefs)
+
+def construct():
+ yml = {}
+ yml['paths'] = {}
+ yml['components'] = {}
+ apidir = os.path.abspath("%s/../../pages/" % bpath)
+ print("Scanning %s" % apidir)
+ for d in os.listdir(apidir):
+ cdir = os.path.abspath("%s/%s" % (apidir, d))
+ if os.path.isdir(cdir):
+ print("Scanning %s" % cdir)
+ for fname in os.listdir(cdir):
+ if fname.endswith(".py"):
+ fpath = "%s/%s" % (cdir, fname)
+ print("Scanning %s" % fpath)
+ contents = open(fpath, "r").read()
+ m = re.search(r"OPENAPI-URI: (\S+)\n##+\n([\s\S]+?)##+", contents)
+ if m:
+ apath = m.group(1)
+ cyml = m.group(2)
+ print("Weaving in API path %s" % apath)
+ cyml = "\n".join([line[2:] for line in cyml.split("\n")])
+ defs = yaml.load(cyml)
+ yml['paths'][apath] = defs
+ else:
+ fname = d
+ if fname.endswith(".py"):
+ fpath = "%s/%s" % (apidir, fname)
+ print("Scanning %s" % fpath)
+ contents = open(fpath, "r").read()
+ m = re.search(r"OPENAPI-URI: (\S+)\n##+\n([\s\S]+?)##+", contents)
+ if m:
+ apath = m.group(1)
+ cyml = m.group(2)
+ print("Weaving in API path %s" % apath)
+ cyml = "\n".join([line[2:] for line in cyml.split("\n")])
+ defs = yaml.load(cyml)
+ yml['paths'][apath] = defs
+ apidir = os.path.abspath("%s/components" % bpath)
+ print("Scanning %s" % apidir)
+ for d in os.listdir(apidir):
+ cdir = os.path.abspath("%s/%s" % (apidir, d))
+ if os.path.isdir(cdir):
+ print("Scanning %s" % cdir)
+ for fname in os.listdir(cdir):
+ if fname.endswith(".yaml"):
+ yml['components'][d] = yml['components'].get(d, {})
+ fpath = "%s/%s" % (cdir, fname)
+ print("Scanning %s" % fpath)
+ defs = yaml.load(open(fpath))
+ yml['components'][d][fname.replace(".yaml", "")] = defs
+ ypath = os.path.abspath("%s/../openapi.yaml" % bpath)
+ with open(ypath, "w") as f:
+ f.write(baseyaml)
+ f.write(yaml.dump(yml, default_flow_style=False))
+ f.close()
+ print("All done!")
+
+if len(sys.argv) > 1 and sys.argv[1] == 'deconstruct':
+ deconstruct()
+else:
+ construct()
diff --git a/api/yaml/openapi/components/schemas/ActionCompleted.yaml b/api/yaml/openapi/components/schemas/ActionCompleted.yaml
new file mode 100644
index 0000000..41f4d54
--- /dev/null
+++ b/api/yaml/openapi/components/schemas/ActionCompleted.yaml
@@ -0,0 +1,10 @@
+########################################################################
+# ActionCompleted #
+########################################################################
+properties:
+ message:
+ description: Acknowledgement message
+ example: Action completed
+ type: string
+required:
+- message
diff --git a/api/yaml/openapi/components/schemas/Empty.yaml b/api/yaml/openapi/components/schemas/Empty.yaml
new file mode 100644
index 0000000..5092518
--- /dev/null
+++ b/api/yaml/openapi/components/schemas/Empty.yaml
@@ -0,0 +1,11 @@
+########################################################################
+# Empty #
+########################################################################
+properties:
+ id:
+ description: optional object ID
+ type: string
+ page:
+ description: optional page id
+ type: string
+required: []
diff --git a/api/yaml/openapi/components/schemas/Error.yaml b/api/yaml/openapi/components/schemas/Error.yaml
new file mode 100644
index 0000000..ed7c9d1
--- /dev/null
+++ b/api/yaml/openapi/components/schemas/Error.yaml
@@ -0,0 +1,16 @@
+########################################################################
+# Error #
+########################################################################
+properties:
+ code:
+ description: HTTP Error Code
+ example: 403
+ format: int16
+ type: integer
+ reason:
+ description: Human readable error message
+ example: You need to be logged in to view this endpoint!
+ type: string
+required:
+- code
+- reason
diff --git a/api/yaml/openapi/components/schemas/Timeseries.yaml b/api/yaml/openapi/components/schemas/Timeseries.yaml
new file mode 100644
index 0000000..63d64be
--- /dev/null
+++ b/api/yaml/openapi/components/schemas/Timeseries.yaml
@@ -0,0 +1,12 @@
+########################################################################
+# Timeseries #
+########################################################################
+properties:
+ interval:
+ type: string
+ okay:
+ type: boolean
+ timeseries:
+ items:
+ $ref: '#/components/schemas/TimeseriesObject'
+ type: array
diff --git a/api/yaml/openapi/components/schemas/TimeseriesObject.yaml b/api/yaml/openapi/components/schemas/TimeseriesObject.yaml
new file mode 100644
index 0000000..850f450
--- /dev/null
+++ b/api/yaml/openapi/components/schemas/TimeseriesObject.yaml
@@ -0,0 +1,18 @@
+########################################################################
+# TimeseriesObject #
+########################################################################
+properties:
+ $item:
+ description: A timeseries object
+ example: 50
+ type: integer
+ $otheritem:
+ description: A timeseries object
+ example: 26
+ type: integer
+ date:
+ description: Seconds since UNIX epoch
+ example: 1508273
+ type: integer
+required:
+- date
diff --git a/api/yaml/openapi/components/schemas/UserAccount.yaml b/api/yaml/openapi/components/schemas/UserAccount.yaml
new file mode 100644
index 0000000..30e6341
--- /dev/null
+++ b/api/yaml/openapi/components/schemas/UserAccount.yaml
@@ -0,0 +1,20 @@
+########################################################################
+# UserAccount #
+########################################################################
+properties:
+ displayname:
+ description: A display name (e.g. full name) for the account
+ example: Warble User
+ type: string
+ email:
+ description: Desired username (email address)
+ example: guest@warble.xyz
+ type: string
+ password:
+ description: Desired password for the account
+ example: warbledemo
+ type: string
+required:
+- email
+- password
+- displayname
diff --git a/api/yaml/openapi/components/schemas/UserCredentials.yaml b/api/yaml/openapi/components/schemas/UserCredentials.yaml
new file mode 100644
index 0000000..c6f4fdd
--- /dev/null
+++ b/api/yaml/openapi/components/schemas/UserCredentials.yaml
@@ -0,0 +1,15 @@
+########################################################################
+# UserCredentials #
+########################################################################
+properties:
+ username:
+ description: Username (email?)
+ example: admin
+ type: string
+ password:
+ description: User password
+ example: warbledemo
+ type: string
+required:
+- username
+- password
diff --git a/api/yaml/openapi/components/schemas/WidgetApp.yaml b/api/yaml/openapi/components/schemas/WidgetApp.yaml
new file mode 100644
index 0000000..687ef0f
--- /dev/null
+++ b/api/yaml/openapi/components/schemas/WidgetApp.yaml
@@ -0,0 +1,37 @@
+########################################################################
+# WidgetApp #
+########################################################################
+properties:
+ blocks:
+ description: Size (width) in UI blocks of the app
+ example: 4
+ type: integer
+ datatype:
+ description: The top category of this data
+ example: repo
+ type: string
+ name:
+ description: The title of the widget app
+ example: Widget Title
+ type: string
+ representation:
+ description: The visual representation style of this widget
+ example: donut
+ type: string
+ source:
+ description: The API endpoint to get data from
+ example: code-evolution
+ type: string
+ target:
+ type: string
+ text:
+ description: Text to insert into the widget (if paragraph type widget)
+ type: string
+ type:
+ description: The type of widget
+ example: My Widget
+ type: string
+required:
+- type
+- name
+- blocks
diff --git a/api/yaml/openapi/components/schemas/WidgetDesign.yaml b/api/yaml/openapi/components/schemas/WidgetDesign.yaml
new file mode 100644
index 0000000..a7ab1f2
--- /dev/null
+++ b/api/yaml/openapi/components/schemas/WidgetDesign.yaml
@@ -0,0 +1,10 @@
+########################################################################
+# WidgetDesign #
+########################################################################
+properties:
+ rows:
+ items:
+ $ref: '#/components/schemas/WidgetRow'
+ type: array
+ title:
+ type: string
diff --git a/api/yaml/openapi/components/schemas/WidgetRow.yaml b/api/yaml/openapi/components/schemas/WidgetRow.yaml
new file mode 100644
index 0000000..4d3d97d
--- /dev/null
+++ b/api/yaml/openapi/components/schemas/WidgetRow.yaml
@@ -0,0 +1,10 @@
+########################################################################
+# WidgetRow #
+########################################################################
+properties:
+ children:
+ items:
+ $ref: '#/components/schemas/WidgetApp'
+ type: array
+ name:
+ type: string
diff --git a/api/yaml/openapi/components/schemas/defaultWidgetArgs.yaml b/api/yaml/openapi/components/schemas/defaultWidgetArgs.yaml
new file mode 100644
index 0000000..5facff9
--- /dev/null
+++ b/api/yaml/openapi/components/schemas/defaultWidgetArgs.yaml
@@ -0,0 +1,79 @@
+########################################################################
+# defaultWidgetArgs #
+########################################################################
+properties:
+ author:
+ description: Turns on author view for code results, as opposed to the default
+ committer view
+ type: boolean
+ collapse:
+ description: for some widgets, this collapses sources based on a regex
+ type: string
+ email:
+ description: filter sources based on an email (a person)
+ type: string
+ from:
+ description: If specified, compile data from this timestamp onwards
+ example: 1400000000
+ type: integer
+ interval:
+ description: If fetching histograms, this specifies the interval to pack figures
+ into. Can be day, week, month, quarter or year
+ example: month
+ type: string
+ links:
+ description: for relationship maps, this denotes the minimum link strength (no.
+ of connections) that makes up a link.
+ type: integer
+ page:
+ type: string
+ quick:
+ description: Turns on quick data for some endpoints, returning only sparse data
+ (thus less traffic)
+ example: false
+ type: boolean
+ search:
+ description: for some widgets, this enables sub-filtering based on searches
+ type: string
+ source:
+ description: If specified, only compile data on this specific sourceID
+ example: abcdef12345678
+ type: string
+ sources:
+ description: for some widget, this fetches all sources
+ type: boolean
+ span:
+ description: For factor charts, denotes the number of months to base factors on
+ from
+ example: 2
+ type: integer
+ subfilter:
+ description: Quickly defined view by sub-filtering the existing view and matching
+ on sourceURLs
+ type: string
+ to:
+ description: If specified, only compile data up until here
+ example: 1503483273
+ type: integer
+ types:
+ description: If set, only return data from sources matching these types
+ example:
+ - jira
+ - bugzilla
+ type: array
+ unique:
+ description: Only compile data from unique commits, ignore duplicates
+ type: boolean
+ view:
+ description: ID Of optional view to use
+ example: abcdef12345678
+ type: string
+ distinguish:
+ description: Enables distinguishing different types of data objects, subject to the individual API endpoint
+ type: boolean
+ example: false
+ relative:
+ description: Enables relative comparison mode for API endpoints that have this feature.
+ type: boolean
+ example: false
+
\ No newline at end of file
diff --git a/api/yaml/openapi/components/securitySchemes/APIKeyAuth.yaml b/api/yaml/openapi/components/securitySchemes/APIKeyAuth.yaml
new file mode 100644
index 0000000..fa29952
--- /dev/null
+++ b/api/yaml/openapi/components/securitySchemes/APIKeyAuth.yaml
@@ -0,0 +1,6 @@
+########################################################################
+# APIKeyAuth #
+########################################################################
+in: headers
+name: APIKey
+type: apiKey
diff --git a/api/yaml/openapi/components/securitySchemes/cookieAuth.yaml b/api/yaml/openapi/components/securitySchemes/cookieAuth.yaml
new file mode 100644
index 0000000..c5b9d51
--- /dev/null
+++ b/api/yaml/openapi/components/securitySchemes/cookieAuth.yaml
@@ -0,0 +1,6 @@
+########################################################################
+# cookieAuth #
+########################################################################
+in: cookie
+name: warble_session
+type: apiKey
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@warble.apache.org
For additional commands, e-mail: commits-help@warble.apache.org
[incubator-warble-server] 07/07: start on a basic readme
Posted by hu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
humbedooh pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-warble-server.git
commit 2fbc64198a34b2a0c7118e9ba956b1b5203a286a
Author: Daniel Gruno <hu...@apache.org>
AuthorDate: Sun Jun 24 22:10:50 2018 -0500
start on a basic readme
---
README.md | 12 +++++++++++-
1 file changed, 11 insertions(+), 1 deletion(-)
diff --git a/README.md b/README.md
index 0e6f749..5036ec9 100644
--- a/README.md
+++ b/README.md
@@ -1,3 +1,13 @@
-# Apache Warble Server Package
+# Apache Warble (incubating) Server Package
This is going to be the master server for Apache Warble (incubating).
+
+## Setup instructions:
+
+* download Warble Server (or clone if you dare!)
+* run `python3 setup/setup.py` and follow the instructions
+* fire up the main application as WSGI, for instance via gunicorn:
+* * `cd /path/to/warble/api`
+* * `gunicorn -w 10 -b 127.0.0.1:8000 handler:application -t 120 -D`
+
+
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@warble.apache.org
For additional commands, e-mail: commits-help@warble.apache.org
[incubator-warble-server] 02/07: separate sessions and accounts
Posted by hu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
humbedooh pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-warble-server.git
commit 76b75a6ad61c93a6154421347409f47570335231
Author: Daniel Gruno <hu...@apache.org>
AuthorDate: Sun Jun 24 21:07:59 2018 -0500
separate sessions and accounts
---
setup/dbs.yaml | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/setup/dbs.yaml b/setup/dbs.yaml
index 7d0dd64..539d269 100644
--- a/setup/dbs.yaml
+++ b/setup/dbs.yaml
@@ -3,10 +3,18 @@ accounts:
driver: sqlite
path: accounts.db
layout:
- cookie: text # Web cookie
userid: text primary key # user ID
password: text # password digest
superuser: boolean # admin or not?
+
+# UI sessions
+sessions:
+ driver: sqlite
+ path: sessions.db
+ layout:
+ cookie: text # HTTP cookie
+ userid: text # corresponding user account
+ timestamp: integer # cookie use timestamp (for timing out sessions)
# Node registry database setup
registry:
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@warble.apache.org
For additional commands, e-mail: commits-help@warble.apache.org
[incubator-warble-server] 01/07: Initial checkout: a setup script!!
Posted by hu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
humbedooh pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-warble-server.git
commit 5f0c3f6d3efba041d21a8244e5b6f1d11a2dceb6
Author: Daniel Gruno <hu...@apache.org>
AuthorDate: Sun Jun 24 15:10:43 2018 -0500
Initial checkout: a setup script!!
More to come later
---
setup/dbs.yaml | 26 ++++++++++
setup/setup.py | 122 +++++++++++++++++++++++++++++++++++++++++++++++
setup/warble.yaml.sample | 9 ++++
3 files changed, 157 insertions(+)
diff --git a/setup/dbs.yaml b/setup/dbs.yaml
new file mode 100644
index 0000000..7d0dd64
--- /dev/null
+++ b/setup/dbs.yaml
@@ -0,0 +1,26 @@
+# Master UI account setup
+accounts:
+ driver: sqlite
+ path: accounts.db
+ layout:
+ cookie: text # Web cookie
+ userid: text primary key # user ID
+ password: text # password digest
+ superuser: boolean # admin or not?
+
+# Node registry database setup
+registry:
+ driver: sqlite
+ path: nodes.db
+ layout:
+ id: integer primary key # ID of node
+ hostname: text # hostname of node
+ apikey: text # API key for requests
+ pubkey: text # public key for encryption/certification
+ verified: boolean # Whether we have verified (accepted) this node via UI
+ enabled: boolean # enabled/disabled
+ description: text # Optional description of node
+ location: text # Physical location of node (addr or DC)
+ ip: text # Known public IP of node
+ lastping: integer # Last time node was alive
+
diff --git a/setup/setup.py b/setup/setup.py
new file mode 100644
index 0000000..b8ed32a
--- /dev/null
+++ b/setup/setup.py
@@ -0,0 +1,122 @@
+#!/usr/bin/env python3
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+WARBLE_VERSION = '0.1.0' # ABI/API compat demarcation.
+WARBLE_DB_VERSION = 1 # First DB version evah!
+
+import sys
+
+if sys.version_info <= (3, 3):
+ print("This script requires Python 3.4 or higher")
+ sys.exit(-1)
+
+import os
+import sys
+import yaml
+import bcrypt
+import sqlite3
+
+# Figure out absolute setup and api paths
+setup_path = os.path.dirname(os.path.realpath(__file__))
+api_path = os.path.realpath("%s/../api" % setup_path)
+db_path = os.path.realpath("%s/../db" % setup_path)
+
+# Open the DB setup instructions and sample warble server config
+dbsetup = yaml.load(open("%s/dbs.yaml" % setup_path))
+myyaml = yaml.load(open("%s/warble.yaml.sample" % setup_path))
+
+# Set up database path
+datapath = input("Where would you like to put the Warble databases?: [%s]" % db_path).strip()
+if not datapath:
+ datapath = db_path
+print("Creating databases...")
+if not os.path.isdir(datapath):
+ print("Creating directory %s" % datapath)
+ try:
+ os.mkdir(datapath)
+ except Exception as err:
+ print("Could not create database directory: %s" % err)
+ sys.exit(-1)
+
+# Set up individual databases (sqlite only for now!!)
+for name, settings in dbsetup.items():
+ if settings['driver'] == 'sqlite':
+ db_filepath = os.path.join(datapath, settings['path'])
+ if os.path.exists(db_filepath):
+ print("Database %s already exists, skipping!" % db_filepath)
+ continue
+ else:
+ print("Creating database file %s" % db_filepath)
+ conn = sqlite3.connect(db_filepath)
+ cursor = conn.cursor()
+ # create a CREATE statement from the yaml layout
+ cstate = "CREATE TABLE %s (%s)" % (
+ name,
+ ", ".join( ["%s %s" % (k, v) for k, v in settings['layout'].items()] )
+ )
+ cursor.execute(cstate)
+ conn.commit()
+ conn.close()
+
+# Generate a super user
+supername = input("Please enter the username of the primary super user to create [admin]: ").strip()
+if not supername:
+ supername = 'admin'
+superpass = input("Please enter the password for the super user account: ").strip()
+
+# Digest pass with bcrypt
+salt = bcrypt.gensalt()
+pwd = bcrypt.hashpw(superpass.encode('utf-8'), salt).decode('ascii')
+
+# Save user to new DB
+db_filepath = os.path.join(datapath, 'accounts.db')
+conn = sqlite3.connect(db_filepath)
+c = conn.cursor()
+try:
+ c.execute("INSERT INTO `accounts` (`cookie`, `userid`, `password`, `superuser`) VALUES ('zzz', ?, ?, 1)", (supername, pwd))
+ conn.commit()
+ print("Saved new super user account %s in %s" % (supername, db_filepath))
+except sqlite3.IntegrityError:
+ print("WARNING: Could not add super user - is the user already in the DB?!")
+conn.close()
+
+
+# Write new warble yaml config
+warble_yaml_path = '%s/yaml/warble.yaml' % api_path
+print("Writing API server config to %s" % warble_yaml_path)
+if os.path.exists(warble_yaml_path):
+ print("WARNING: Warble YAML already exists on disk, skipping this step!!")
+else:
+ myconfig = {
+ 'database': {
+ 'version': WARBLE_DB_VERSION,
+ 'driver': 'sqlite',
+ 'path': datapath
+ },
+ 'mail': {
+ 'host': 'localhost',
+ 'port': 25,
+ 'sender': 'Apache Warble<wa...@demo.warble.xyz>'
+ }
+ }
+ with open(warble_yaml_path, "w") as f:
+ f.write(yaml.dump(myconfig, default_flow_style = False))
+ f.close()
+
+
+print("All done, Warble should...work now :)")
+print("If needed, you can fine tune %s to suit your needs." % warble_yaml_path)
+
diff --git a/setup/warble.yaml.sample b/setup/warble.yaml.sample
new file mode 100644
index 0000000..4533136
--- /dev/null
+++ b/setup/warble.yaml.sample
@@ -0,0 +1,9 @@
+database:
+ driver: sqlite
+ path: /foo/bar/baz
+
+mail:
+ mailhost: localhost
+ mailport: 25
+ sender: Apache Warble <wa...@example.org>
+
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@warble.apache.org
For additional commands, e-mail: commits-help@warble.apache.org
[incubator-warble-server] 05/07: generate openapi yaml
Posted by hu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
humbedooh pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-warble-server.git
commit 2fa6871d9051688695d83a54a4f3ba38771cae4d
Author: Daniel Gruno <hu...@apache.org>
AuthorDate: Sun Jun 24 22:06:03 2018 -0500
generate openapi yaml
---
api/yaml/openapi.yaml | 403 ++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 403 insertions(+)
diff --git a/api/yaml/openapi.yaml b/api/yaml/openapi.yaml
new file mode 100644
index 0000000..b3de7b6
--- /dev/null
+++ b/api/yaml/openapi.yaml
@@ -0,0 +1,403 @@
+
+# THIS IS PULLED FROM SCRIPTS AND AUTOGENERATED!
+# Please use openapi/combine.py to regenerate!
+openapi: 3.0.0
+info:
+ version: 1.0.0
+ description: This is the API specifications for interacting with the Warble Server.
+ title: Apache Warble API
+ license:
+ name: Apache 2.0
+ url: 'http://www.apache.org/licenses/LICENSE-2.0.html'
+components:
+ schemas:
+ ActionCompleted:
+ properties:
+ message:
+ description: Acknowledgement message
+ example: Action completed
+ type: string
+ required:
+ - message
+ Empty:
+ properties:
+ id:
+ description: optional object ID
+ type: string
+ page:
+ description: optional page id
+ type: string
+ required: []
+ Error:
+ properties:
+ code:
+ description: HTTP Error Code
+ example: 403
+ format: int16
+ type: integer
+ reason:
+ description: Human readable error message
+ example: You need to be logged in to view this endpoint!
+ type: string
+ required:
+ - code
+ - reason
+ Timeseries:
+ properties:
+ interval:
+ type: string
+ okay:
+ type: boolean
+ timeseries:
+ items:
+ $ref: '#/components/schemas/TimeseriesObject'
+ type: array
+ TimeseriesObject:
+ properties:
+ $item:
+ description: A timeseries object
+ example: 50
+ type: integer
+ $otheritem:
+ description: A timeseries object
+ example: 26
+ type: integer
+ date:
+ description: Seconds since UNIX epoch
+ example: 1508273
+ type: integer
+ required:
+ - date
+ UserAccount:
+ properties:
+ displayname:
+ description: A display name (e.g. full name) for the account
+ example: Warble User
+ type: string
+ email:
+ description: Desired username (email address)
+ example: guest@warble.xyz
+ type: string
+ password:
+ description: Desired password for the account
+ example: warbledemo
+ type: string
+ required:
+ - email
+ - password
+ - displayname
+ UserCredentials:
+ properties:
+ password:
+ description: User password
+ example: warbledemo
+ type: string
+ username:
+ description: Username (email?)
+ example: admin
+ type: string
+ required:
+ - username
+ - password
+ WidgetApp:
+ properties:
+ blocks:
+ description: Size (width) in UI blocks of the app
+ example: 4
+ type: integer
+ datatype:
+ description: The top category of this data
+ example: repo
+ type: string
+ name:
+ description: The title of the widget app
+ example: Widget Title
+ type: string
+ representation:
+ description: The visual representation style of this widget
+ example: donut
+ type: string
+ source:
+ description: The API endpoint to get data from
+ example: code-evolution
+ type: string
+ target:
+ type: string
+ text:
+ description: Text to insert into the widget (if paragraph type widget)
+ type: string
+ type:
+ description: The type of widget
+ example: My Widget
+ type: string
+ required:
+ - type
+ - name
+ - blocks
+ WidgetDesign:
+ properties:
+ rows:
+ items:
+ $ref: '#/components/schemas/WidgetRow'
+ type: array
+ title:
+ type: string
+ WidgetRow:
+ properties:
+ children:
+ items:
+ $ref: '#/components/schemas/WidgetApp'
+ type: array
+ name:
+ type: string
+ defaultWidgetArgs:
+ properties:
+ author:
+ description: Turns on author view for code results, as opposed to the default
+ committer view
+ type: boolean
+ collapse:
+ description: for some widgets, this collapses sources based on a regex
+ type: string
+ distinguish:
+ description: Enables distinguishing different types of data objects, subject
+ to the individual API endpoint
+ example: false
+ type: boolean
+ email:
+ description: filter sources based on an email (a person)
+ type: string
+ from:
+ description: If specified, compile data from this timestamp onwards
+ example: 1400000000
+ type: integer
+ interval:
+ description: If fetching histograms, this specifies the interval to pack
+ figures into. Can be day, week, month, quarter or year
+ example: month
+ type: string
+ links:
+ description: for relationship maps, this denotes the minimum link strength
+ (no. of connections) that makes up a link.
+ type: integer
+ page:
+ type: string
+ quick:
+ description: Turns on quick data for some endpoints, returning only sparse
+ data (thus less traffic)
+ example: false
+ type: boolean
+ relative:
+ description: Enables relative comparison mode for API endpoints that have
+ this feature.
+ example: false
+ type: boolean
+ search:
+ description: for some widgets, this enables sub-filtering based on searches
+ type: string
+ source:
+ description: If specified, only compile data on this specific sourceID
+ example: abcdef12345678
+ type: string
+ sources:
+ description: for some widget, this fetches all sources
+ type: boolean
+ span:
+ description: For factor charts, denotes the number of months to base factors
+ on from
+ example: 2
+ type: integer
+ subfilter:
+ description: Quickly defined view by sub-filtering the existing view and
+ matching on sourceURLs
+ type: string
+ to:
+ description: If specified, only compile data up until here
+ example: 1503483273
+ type: integer
+ types:
+ description: If set, only return data from sources matching these types
+ example:
+ - jira
+ - bugzilla
+ type: array
+ unique:
+ description: Only compile data from unique commits, ignore duplicates
+ type: boolean
+ view:
+ description: ID Of optional view to use
+ example: abcdef12345678
+ type: string
+ securitySchemes:
+ APIKeyAuth:
+ in: headers
+ name: APIKey
+ type: apiKey
+ cookieAuth:
+ in: cookie
+ name: warble_session
+ type: apiKey
+paths:
+ /api/account:
+ delete:
+ requestBody:
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/UserName'
+ description: User ID
+ required: true
+ responses:
+ '200':
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/ActionCompleted'
+ description: 200 response
+ default:
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/Error'
+ description: unexpected error
+ security:
+ - cookieAuth: []
+ summary: Delete an account
+ patch:
+ requestBody:
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/UserAccountEdit'
+ description: User credentials
+ required: true
+ responses:
+ '200':
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/ActionCompleted'
+ description: 200 response
+ default:
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/Error'
+ description: unexpected error
+ summary: Edit an account
+ put:
+ requestBody:
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/UserAccount'
+ description: User credentials
+ required: true
+ responses:
+ '200':
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/ActionCompleted'
+ description: 200 response
+ default:
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/Error'
+ description: unexpected error
+ summary: Create a new account
+ /api/session:
+ delete:
+ requestBody:
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/Empty'
+ description: Nada
+ required: true
+ responses:
+ '200':
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/ActionCompleted'
+ description: Logout successful
+ default:
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/Error'
+ description: unexpected error
+ security:
+ - cookieAuth: []
+ summary: Log out (remove session)
+ get:
+ responses:
+ '200':
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/UserData'
+ description: 200 response
+ default:
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/Error'
+ description: unexpected error
+ security:
+ - cookieAuth: []
+ summary: Display your login details
+ put:
+ requestBody:
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/UserCredentials'
+ description: User credentials
+ required: true
+ responses:
+ '200':
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/ActionCompleted'
+ description: Login successful
+ headers:
+ Set-Cookie:
+ schema:
+ example: 77488a26-23c2-4e29-94a1-6a0738f6a3ff
+ type: string
+ default:
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/Error'
+ description: unexpected error
+ summary: Log in
+ /api/widgets/{pageid}:
+ get:
+ parameters:
+ - description: Page ID to fetch design for
+ in: path
+ name: pageid
+ required: true
+ schema:
+ type: string
+ responses:
+ '200':
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/WidgetDesign'
+ description: 200 Response
+ default:
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/Error'
+ description: unexpected error
+ security:
+ - cookieAuth: []
+ summary: Shows the widget layout for a specific page
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@warble.apache.org
For additional commands, e-mail: commits-help@warble.apache.org
[incubator-warble-server] 03/07: no longer need cookie var here
Posted by hu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
humbedooh pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-warble-server.git
commit 80def088bcce6b45b1b27b35404f0a9ae35fbd64
Author: Daniel Gruno <hu...@apache.org>
AuthorDate: Sun Jun 24 22:03:45 2018 -0500
no longer need cookie var here
---
setup/setup.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/setup/setup.py b/setup/setup.py
index b8ed32a..87c59cc 100644
--- a/setup/setup.py
+++ b/setup/setup.py
@@ -86,7 +86,7 @@ db_filepath = os.path.join(datapath, 'accounts.db')
conn = sqlite3.connect(db_filepath)
c = conn.cursor()
try:
- c.execute("INSERT INTO `accounts` (`cookie`, `userid`, `password`, `superuser`) VALUES ('zzz', ?, ?, 1)", (supername, pwd))
+ c.execute("INSERT INTO `accounts` (`userid`, `password`, `superuser`) VALUES (?, ?, 1)", (supername, pwd))
conn.commit()
print("Saved new super user account %s in %s" % (supername, db_filepath))
except sqlite3.IntegrityError:
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@warble.apache.org
For additional commands, e-mail: commits-help@warble.apache.org
[incubator-warble-server] 06/07: add main gitignore
Posted by hu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
humbedooh pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-warble-server.git
commit 1c06186ffa3406e81aac76b265fa18e3fb53d1e3
Author: Daniel Gruno <hu...@apache.org>
AuthorDate: Sun Jun 24 22:06:49 2018 -0500
add main gitignore
ignore db dir and such
---
.gitignore | 3 +++
1 file changed, 3 insertions(+)
diff --git a/.gitignore b/.gitignore
new file mode 100644
index 0000000..84a7554
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1,3 @@
+__pycache__
+venv
+db
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@warble.apache.org
For additional commands, e-mail: commits-help@warble.apache.org