You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@brooklyn.apache.org by neykov <gi...@git.apache.org> on 2014/07/24 12:47:30 UTC
[GitHub] incubator-brooklyn pull request: Versioning catalog items
GitHub user neykov opened a pull request:
https://github.com/apache/incubator-brooklyn/pull/92
Versioning catalog items
Subtasks:
* Version bundles in brooklyn.catalog section
* Version catalog items
* Incorporate new version info in REST API
* Display version info in GUI
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/neykov/incubator-brooklyn catalog-versions
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-brooklyn/pull/92.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #92
----
commit 340f14a390b4909e4bf2f684a1129674a628eaf5
Author: Svetoslav Neykov <sv...@cloudsoftcorp.com>
Date: 2014-07-17T14:40:41Z
Catalog items versioning
The deprecated methods without version info will use the default 0.0.0 version.
The version attribute for yaml items is required now.
commit e9f68983bbea99c5729534842ad4e6453d5f9a0a
Author: Svetoslav Neykov <sv...@cloudsoftcorp.com>
Date: 2014-07-17T15:37:42Z
REST API catalog versioning
commit a091afeb1764f571643af527b9454051fd40f26a
Author: Svetoslav Neykov <sv...@cloudsoftcorp.com>
Date: 2014-07-18T16:32:32Z
jsgui versioning support for catalog items
commit d48ba95fa477c765a8ffc503a7359e92c54b222e
Author: Svetoslav Neykov <sv...@cloudsoftcorp.com>
Date: 2014-07-21T16:08:02Z
Support name+version when referencing OSGi bundles in YAML.
commit d3fb018d993a2a72b243b0126e8388301f40488d
Author: Svetoslav Neykov <sv...@cloudsoftcorp.com>
Date: 2014-07-23T20:44:03Z
Don't allow updating existing catalog items.
Catalog items should be immutable.
commit 1e957836c01ec9f2287b2aba10417e39386f05bf
Author: Svetoslav Neykov <sv...@cloudsoftcorp.com>
Date: 2014-07-24T09:29:31Z
Using AssemblyTemplate.getName() for getting catalog items is wrong.
Catalog items are registered by id+version in the brooklyn.config section,
the name is unrelated which means that the items were never found.
The code path in BrooklynEntityMatcher is not needed anyway, already
handled by the instantiator.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Versioning catalog items
Posted by ahgittin <gi...@git.apache.org>.
Github user ahgittin commented on the pull request:
https://github.com/apache/incubator-brooklyn/pull/92#issuecomment-61643946
After a lot of thought here is my thinking:
**Everything in catalog gets a version number**
* If version is not supplied on a catalog addition, normally:
* If no versions are present, we set `0.0.0`
* If the only versions that are present are `0.0.0` or `0.0.0_N` we set `0.0.0_M` where `M` is one more than the largest such `N` (i.e. so we monotonically increase a qualifier)
* If other versions are present, we fail
* If a version *is* supplied but it is already present, normally we apply the monotonic increasing strategy above
* If a version is supplied and is not present, we add it
* The exceptions to the *normally* items above are that if a `force` parameter is supplied with a version number, we will set it always, replacing items which may exist; if `force` is supplied without a version number, we fail.
This allows us to have unique explicit versions for everything, unless a user has forced something (which would only be done for cleanup.)
**There is a preferred/default version**
So references *may* specify a version but do not have to. References to specific versions should follow the `catalogItemId` nomenclature of `my-catalog-item-symbolic-name:0.0.0`. (In some contexts a separate `version` key might be supported, but it might be better not to.)
A user could set this by including `isDefaultVersion: true` when adding. This tag would be interpreted specially: it would be persisted, but it would also have the effect of clearing any such tag in any other version. `isDefaultVersion: false` would also be permitted, and it would prevent a version from being available as default.
To determine the default/preferred version at runtime, the system chooses:
* The unique entry with `isDefaultVersion: true`, if there is one (if there is multiple it should probably clear lower numbers and log a warning ... probably means someone has been mucking with catalog files); otherwise
* The highest non-SNAPSHOT version which does not have `isDefaultValue: false` set, if present; otherwise
* The highest version which does not have `isDefaultValue: false` set; otherwise
* It reports that no such entity is available
**As for data structures...**
The field `catalogItemId` in many places (e.g. in `Entity`) should always be in the format `symbolic-name:version`.
`CatalogItem` instances themsleves should be updated to have the following fields to identify them:
* 'id' -- the random ID field exists (it is inherited) but never used; it might be persisted which could be useful for some types of tracking; `getId()` is overwritten to return `getCatalogItemId()`
* `catalogItemId` -- this should always be set to `symbolicName+":"+version` whenever either of those two fields changes; never `null`
* 'symbolicName` -- cf OSGi property, this is the version-less name of the item; it replaces `registeredTypeName`; never `null`
* `version` -- this is an OSGi-style version (maven style also accepted); treated as semver; never `null`
* `displayName` -- this is a human-readable name; it is displayed in the catalog, along with `catalogItemId`; if `null`, the getter returns `catalogItemId`; it should replace `name` in most places (though in some places `catalogItemId` or `symbolicName` should perhaps be used)
(Other fields e.g. `javaType` and `description` continue exist but may be null, although at least one of a `javaType` or a `yamlPlan` is required.)
I think we will have to keep the old fields due to persistence but let's deprecate them; if they can be migrated on rebind to the new fields that would be great as it makes it easier to mark the old ones as transient and remove them in time.
**On the API and YAML**
Versions may be specified either using `name: my-catalog-item-symbolic-name:0.0.0` or `{ name: my-catalog-item-symbolic-name, version: 0.0.0 }`.
Additionally, `symbolicName` or `id` are also accepted as ways to set the symbolic name. `name` has lower priority and is used also to set `displayName`. Some rules:
* If `symbolicName` *and* `id` are specified, it is an error if they are not identical.
* If `name`, `displayName`, and either `symbolicName` or `id` is specified, we warn that `name` is being ignored
* If `symbolicName` (or `id`, or `name` in the absence of the other two) contains a `:`, everything after the `:` is interpreted as a version, and must be valid as per semver/osgi/maven regexing. The internal `symbolicName` does *not* store the version, whereas the `catalogItemId` *does*. If an additional `version` key is supplied, its value must match identically.
There should be an API call to set a default version.
On deletion, a version is required. If a `catalogItemId`is in use, an error is thrown, unless `force` is specified, in which case the item is deleted (and corresponding entities will not rebind unless it is added back).
---
Does this hang together? Does it seem right? @aledsage @neykov
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Versioning catalog items
Posted by neykov <gi...@git.apache.org>.
Github user neykov commented on the pull request:
https://github.com/apache/incubator-brooklyn/pull/92#issuecomment-63432702
Changes implemented in https://github.com/apache/incubator-brooklyn/pull/312
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Versioning catalog items
Posted by ahgittin <gi...@git.apache.org>.
Github user ahgittin commented on the pull request:
https://github.com/apache/incubator-brooklyn/pull/92#issuecomment-61803962
Good discussion just now with @aledsage and @neykov . Conclusions include some changes to the above proposal:
* When no version is specified, call it `0.0.0_SNAPSHOT`.
* Not necessary now to apply monotonic increasing qualifier on conflict, and potentially surprising. We should just fail if the version exists. If it is easy in selected cases (viz. `0.0.0_SNAPSHOT` or perhaps all items `*_SNAPSHOT*`), monotonic increasing qualifier could be appended there automatically on conflict.
* We do not need to support setting a default version on the catalog for now. The highest version (preferring non-snapshot) is used when a caller does not request a version.
* In YAML we will support the fields above (viz. including `name` in addition to `symbolicName`, `version`, `id`, and `displayName`), but in our examples we will follow a consistent approach, viz. `id`. The others (in particular `name`) might be deprecated in the future to minimize the number of ways of doing this. (Or we may experiment and discover that `name` is actually really nice, or all these ays are nice, and then keep this status quo or deprecate some of the other items, TBD!)
Other tasks TODO:
* Test whether we can do anything for OSGi multiple instances of the same version. I thought this was the behaviour but turns out if you have two bundles with the same `name:version`, we use any of them, irrespective of the defining URL. If we can differentiate between them based on URL we should. (Alex to check.)
* Write documentation for adding to catalog, with examples, and setting versions and how those work. Include `force`, and the importance of OSGi versioning and behaviour there (including the fact that restart might pick up different bundles, in good ways and bad.)
* After this task, we'll need a way to update entities from one catalog item id version to another. (I could see this as supplying a map of catalog item id regex->target which should be upgraded, discovering the matching entities, selecting all/any of them, and then deserializing and reserializing
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Versioning catalog items
Posted by neykov <gi...@git.apache.org>.
Github user neykov closed the pull request at:
https://github.com/apache/incubator-brooklyn/pull/92
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Versioning catalog items
Posted by ahgittin <gi...@git.apache.org>.
Github user ahgittin commented on the pull request:
https://github.com/apache/incubator-brooklyn/pull/92#issuecomment-50615428
I think we need some way of finding a default. Without this existing YAML blueprints break, the simple "add app" from a catalog breaks, and users have to know blueprint versions to do anything.
Based on external discussions, my suggestions are to support both versionless items and latest-semver-release semantics. Specifically:
* Where an entity is registered/available without version information (explicitly added to catalog without version or picked up from java classes) we:
* Use it when the user has not suggested version
* Allow it to be replaced if an updated item is added with the same name/id and no version
* Where a catalog item is added without version information we refer to it as an "alias" and recommend that it point via YAML to a versioned item
* Where an entity is registered/available with version information:
* If the user specifies a version, we must match it exactly (and in time, we may support a version range from the user)
* If the user does not specify a version, we look for an "alias" with that ID (ie no version information), if there is none we look for the most recent release version (as per semver), and if there is still none we look for a java class with that name
* We require an extra "force" parameter if someone wants to replace it (with the same version), or if they want to delete it when it is in use
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Versioning catalog items
Posted by ahgittin <gi...@git.apache.org>.
Github user ahgittin commented on the pull request:
https://github.com/apache/incubator-brooklyn/pull/92#issuecomment-51740421
i think we agree at the two enpoints -- from user side, they are *not* required to supply a version; and in back-end everything *does* have a version wherever possible. so if user specifies "MySoftware" brooklyn internally srtores that it is MySoftware_1.0.1 or whatever.
it is the mapping in between that we are unsure of. i don't like the 999.0.0 idea -- it is nice in some ways but feels an ugly "magic numbers" hack in others. i also don't think that the intermediate/alias needs to be versioned -- as that is *not* what is stored (ie the alias is resolved at deploy time and not used anymore thereafter). that should simplify things a big?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Versioning catalog items
Posted by neykov <gi...@git.apache.org>.
Github user neykov commented on the pull request:
https://github.com/apache/incubator-brooklyn/pull/92#issuecomment-50636154
These are all great suggestions, it seems that the consensus in the area (Chef, Puppet, et al, even OSGi) allows for requesting an item without explicitly specifying a version. This allows the users not to care about versioning until they really need to.
Specifically for creating items, isn't it cleaner (from a usage perspective) that we don't support versionless items? They make the behaviour too overloaded. Of course existing items should be supported but the behaviour deprecated and removed in a subsequent version.
I think items "picked up from java classes" should be versioned as well. If coming from an OSGi bundle then it would be great if we could retrieve the version from the bundle but I don't this is possible.
Having immutable catalog items seems like a desired goal especially when we have them serialized and mirrored by other nodes. It would be clear how an existing instance was created even if the catalog items it relies on were updated to a newer version and aliases updated.
It seems that your suggestion still could fit in an immutable catalog which doesn't allow versionless items:
* for existing items without version create them with version 0.0.0
* require explicit versioning for new items as per semver
* the alias (items with explicitly set version: default) could have version 999.0.0
* when an item is referenced without a version use the latest stable version for that id
* there could be an exception to update the catalog items for legacy and default versions or better yet, automatically increment the version (i.e. 0.0.1 for the next update of a legacy item without version, 999.0.1 for the next update of a default item)
* no need to support force
* the only immutable operation would be delete, which as a side effect would allow a subsequent addition with the same version, or maybe delete should just be marking it as non-accessible?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---