Re: Migrating our wiki to Confluence

[Johan Corveleyn <jc...@gmail.com>]

On Fri, Nov 24, 2017 at 1:29 PM, Stefan <lu...@posteo.de> wrote:
> On 24/11/2017 12:25, Branko Čibej wrote:
>> On 24.11.2017 12:20, Johan Corveleyn wrote:
>>> At the hackathon we (me, Stefan Hett, Stefan Fuhrmann, Bert, Julian)
>>> discussed migrating our wiki from the current MoinMoin
>>> (wiki.apache.org/subversion) to the ASF Confluence instance
>>> (cwiki.apache.org). This has been discussed / tried before [1], but
>>> let's have another try.
>>>
>>> Motivation:
>>>
>>>   * Make it possible to set up pages that anyone can edit, after
>>> creating an account without interaction with some administrator /
>>> moderator (in moinmoin, a new account needs to be activated by someone
>>> with admin privileges -- too much friction).
>>>     - example that triggered this: let's keep a list of distributors /
>>> packagers, and a list of tool vendors / integrators -- would be nice
>>> if people could add themselves to such a page. Apart from that we want
>>> more people to contribute to all kinds of docs / design / discussion.
>>>
>>>   * Faster editing / saving (moinmoin still has performance problems).
>>>
>>>   * In general: making it easier and nicer for anyone to contribute,
>>> lowering barriers, ...
>>>
>>>   * Additional benefit: user accounts are shared between the ASF JIRA
>>> and Confluence instances.
>>>
>>>
>>> How to do it?
>>>   * Just copy / paste (or export / import) existing pages (keeping the
>>> last author or something), it's no must to keep history.
>>>   * What about existing links to our pages? Can we let them redirect
>>> to the new pages?
>>>   * We should ask around with other ASF projects that have done this.
>>>
>>>
>>> We still need to figure out some details on the "how", but regardless
>>> we feel we should really just go ahead and do this. I'm willing to
>>> spend some time on this myself (help is very welcome of course), if
>>> the community agrees.
>> The most important issues are:
>> * internal links between pages
>> * external links to pages
>>
>> Although if the latter works (with properly configured redirects), the
>> former should, too.
>>
>> But if this can't be guaranteed (and automated), you've just volunteered
>> to do the migration manually. :)
>>
>> -- Brane
>
> The number of pages in the wiki is quite manageable, and I'm more than
> willing to help with the migration process.
>
> Regarding redirects/links: Shouldn't be much an issue to simply set
> explicit redirects to the URLs one by one from the current wiki pages to
> the new locations inside Confluence.
>
> Setting the author of a page needs to be done in the DB directly though,
> so we need to check with Infra whether they'd do it for us (I assume atm
> we are talking roughly 20-50 pages so it's really not much work to
> create an sql-statement for them).

I've tried to get a list of all our current wiki pages. This search
gives 105 pages, but some can be ignored (some system pages, and the
"profile pages" of each user):

https://wiki.apache.org/subversion/FindPage?advancedsearch=1&and_terms=.&mtime=8+years+before&action=fullsearch

Instead of using this search (with mtime=8+years+before or somesuch,
i.e. fill in '8 years before', and a '.' for search terms, in
https://wiki.apache.org/subversion/FindPage), one can also list all
pages (https://wiki.apache.org/subversion/TitleIndex), but that gives
many more irrelevant pages, as it also includes help pages from
MoinMoin itself.

Since I like history (I'm interested in version control after all) I'd
like to try to import them into Confluence with history. There seems
to be a tool to export pages (optionally with history) from MoinMoin,
to convert them into something that can be fed to the "Universal Wiki
Converter" [1]. I'll read a bit more about that, and interface with
infra to see if we can give that a try.

Also, two INFRA jira tickets I found interesting:
* https://issues.apache.org/jira/browse/INFRA-7256 (Create Confluence
wiki for CouchDB) -- this one had a reference to the Wiki Converter
* https://issues.apache.org/jira/browse/INFRA-10942 (Moving Cassandra
wiki to confluence) -- this one mentioned "As for 302 the moin wiki
instance has the ability for you to redirect pages themselves", as
Stefan also suggested above.


[1] https://migrations.atlassian.net/wiki/spaces/UWC/pages/1015848/UWC+MoinMoin+Notes

-- 
Johan

Re: Migrating our wiki to Confluence

[Julian Foad <ju...@apache.org>]

Johan Corveleyn wrote:
> I've tried to get a list of all our current wiki pages. [...]

Thanks for the progress update.

I notice there are several "orphaned" pages that are useful. For 
example, "Berlin2015" was orphaned; I have just linked it from "Meetings".

> Since I like history (I'm interested in version control after all) I'd
> like to try to import them into Confluence with history. There seems
> to be a tool to export pages (optionally with history) from MoinMoin,
> to convert them into something that can be fed to the "Universal Wiki
> Converter" [1]. I'll read a bit more about that, and interface with
> infra to see if we can give that a try.

That would feel good because we like versioned history, although I think 
it is not important so don't hold up the progress because of it.

- Julian


> Also, two INFRA jira tickets I found interesting:
> * https://issues.apache.org/jira/browse/INFRA-7256 (Create Confluence
> wiki for CouchDB) -- this one had a reference to the Wiki Converter
> * https://issues.apache.org/jira/browse/INFRA-10942 (Moving Cassandra
> wiki to confluence) -- this one mentioned "As for 302 the moin wiki
> instance has the ability for you to redirect pages themselves", as
> Stefan also suggested above.
> 
> 
> [1] https://migrations.atlassian.net/wiki/spaces/UWC/pages/1015848/UWC+MoinMoin+Notes

Re: Migrating our wiki to Confluence

[Stefan Hett <st...@egosoft.com>]

On 12/1/2017 5:07 PM, Johan Corveleyn wrote:
> On Fri, Nov 24, 2017 at 1:29 PM, Stefan <lu...@posteo.de> wrote:
>> On 24/11/2017 12:25, Branko Čibej wrote:
>>> On 24.11.2017 12:20, Johan Corveleyn wrote:
>>>> At the hackathon we (me, Stefan Hett, Stefan Fuhrmann, Bert, Julian)
>>>> discussed migrating our wiki from the current MoinMoin
>>>> (wiki.apache.org/subversion) to the ASF Confluence instance
>>>> (cwiki.apache.org). This has been discussed / tried before [1], but
>>>> let's have another try.
>>>>
>>>> Motivation:
>>>>
>>>>    * Make it possible to set up pages that anyone can edit, after
>>>> creating an account without interaction with some administrator /
>>>> moderator (in moinmoin, a new account needs to be activated by someone
>>>> with admin privileges -- too much friction).
>>>>      - example that triggered this: let's keep a list of distributors /
>>>> packagers, and a list of tool vendors / integrators -- would be nice
>>>> if people could add themselves to such a page. Apart from that we want
>>>> more people to contribute to all kinds of docs / design / discussion.
>>>>
>>>>    * Faster editing / saving (moinmoin still has performance problems).
>>>>
>>>>    * In general: making it easier and nicer for anyone to contribute,
>>>> lowering barriers, ...
>>>>
>>>>    * Additional benefit: user accounts are shared between the ASF JIRA
>>>> and Confluence instances.
>>>>
>>>>
>>>> How to do it?
>>>>    * Just copy / paste (or export / import) existing pages (keeping the
>>>> last author or something), it's no must to keep history.
>>>>    * What about existing links to our pages? Can we let them redirect
>>>> to the new pages?
>>>>    * We should ask around with other ASF projects that have done this.
>>>>
>>>>
>>>> We still need to figure out some details on the "how", but regardless
>>>> we feel we should really just go ahead and do this. I'm willing to
>>>> spend some time on this myself (help is very welcome of course), if
>>>> the community agrees.
>>> The most important issues are:
>>> * internal links between pages
>>> * external links to pages
>>>
>>> Although if the latter works (with properly configured redirects), the
>>> former should, too.
>>>
>>> But if this can't be guaranteed (and automated), you've just volunteered
>>> to do the migration manually. :)
>>>
>>> -- Brane
>> The number of pages in the wiki is quite manageable, and I'm more than
>> willing to help with the migration process.
>>
>> Regarding redirects/links: Shouldn't be much an issue to simply set
>> explicit redirects to the URLs one by one from the current wiki pages to
>> the new locations inside Confluence.
>>
>> Setting the author of a page needs to be done in the DB directly though,
>> so we need to check with Infra whether they'd do it for us (I assume atm
>> we are talking roughly 20-50 pages so it's really not much work to
>> create an sql-statement for them).
> I've tried to get a list of all our current wiki pages. This search
> gives 105 pages, but some can be ignored (some system pages, and the
> "profile pages" of each user):
>
> https://wiki.apache.org/subversion/FindPage?advancedsearch=1&and_terms=.&mtime=8+years+before&action=fullsearch
>
> Instead of using this search (with mtime=8+years+before or somesuch,
> i.e. fill in '8 years before', and a '.' for search terms, in
> https://wiki.apache.org/subversion/FindPage), one can also list all
> pages (https://wiki.apache.org/subversion/TitleIndex), but that gives
> many more irrelevant pages, as it also includes help pages from
> MoinMoin itself.
Correct, https://wiki.apache.org/subversion/TitleIndex is the list I 
skimmed through when estimating the number of to be migrated pages. 
Excluding all the moin-wiki-help pages and the ones which seem to not 
work at all (i.e. the Chinese ones for example just give me internal 
server errors) you'd come up with a rough estimate of around 50 pages or 
so. Excluding pages which should presumably be filtered out too (f.e. 
https://wiki.apache.org/subversion/OsieroconeStrony) reduces the amount 
a bit further.
>
> Since I like history (I'm interested in version control after all) I'd
> like to try to import them into Confluence with history. There seems
> to be a tool to export pages (optionally with history) from MoinMoin,
> to convert them into something that can be fed to the "Universal Wiki
> Converter" [1]. I'll read a bit more about that, and interface with
> infra to see if we can give that a try.
>
> Also, two INFRA jira tickets I found interesting:
> * https://issues.apache.org/jira/browse/INFRA-7256 (Create Confluence
> wiki for CouchDB) -- this one had a reference to the Wiki Converter
> * https://issues.apache.org/jira/browse/INFRA-10942 (Moving Cassandra
> wiki to confluence) -- this one mentioned "As for 302 the moin wiki
> instance has the ability for you to redirect pages themselves", as
> Stefan also suggested above.
>
>
> [1] https://migrations.atlassian.net/wiki/spaces/UWC/pages/1015848/UWC+MoinMoin+Notes
>
UWC is what's a usable way to migrate wikis to Conflunece. But the last 
time I reviewed this (which was around the time of the Confluence 4.x 
releases as far as I remember) it suggested it's some work and not an 
all-in-one solution which can easily be applied. Ultimately this is why 
we in our company ended up migrating things manually. If you wanna give 
it a try, spend the required time to work out the details, and Infra 
providing the (presumably) required support to import the data, I'm all 
for going that way instead of simply moving all pages over manually. In 
the end it has been a couple of years since I looked at UWC and things 
might be easier to achieve now than they were back then.

That said, thanks for picking this up, Johan.

-- 
Regards,
Stefan Hett