I would like some feedback on the archive category. but first…
What is the Archive category? It is a new category that will one day contain the ENTIRE old www.chaos.dwarf.com forum content. I have been writing a piece of software that does A LOT to make sure the data stays as consistent as possible.
the tool does the following:
It takes a part (this is configurable) of ALL the posts ever made on the old forum (0.25 million) and loops through them chronologically.
It checks if the post’s category already exists on discourse, if not it is created
it checks if there is a topic for this post, if not it creates the topic (the post then technically is a topic)
If there is a topic it creates a reply in the topic
→ the message is sanitized, meaning the MyBB markup tags that do not work on discourse are removed/replaced.
→ the message is scanned for ALL links that either refer to the old forum (to another topic or something) OR to images that were hosted on the forum itself. These links are stored in a database for later reprocessing, since some posts do not yet exist at this point.
→ The original forums original username is prefixed to the message for continuity’s sake. Usernames are matched by a High probability algorithm so “Pyro Stick” and “Pyrostick” can be matched. This MIGHT not always be 100% correct.
→ Posts by Users that do not exist on this forum are made by the “System” user.
The creation of the post on discourse is logged in a database so i can link the old post id to the new discourse post at any time AND so i always know what post was last processed.
Whenever i feel like it, or maybe ill do it automatically after each bulk transfer, or whenever Ill take those url’s that i saved in the database and do a lookup for the Discourse version of that page. If it’s found the post is updated and saved. The record is marked as complete.
→ This will be extended to also upload the missing images that are physically on my pc but only after storage on discourse has been taken into account.
Now… This is a lot of work to check, and not everything is fixable. A dead external link will still be dead. Some MyBB functionality (wiki linking and such) will not work if the old MyBB plugin is used) and so forth.
If there is anyone who has some spare time would be willing to do some comparing and checking of the archive and the old forums version that would be GREAT!
these posts have had some of their URL’s updated by the replacement code.
for example the first Xander link in the fist post, the pulper link in the second post and the links in the last two posts.
Comparing both forums is fairly simple
open the post on Discourse you want to check
On the old forum you would go to the “Modeling Ideas and Advice” category, and look for the “Compiled Conversions” thread (probably on the last page, but a search on the topic name works well too. )
Think of all the cool inspiration we’re going to find looking through this stuff! I bet @Admiral and @tjub can point us to cool old threads and blogs, too.
And important to note is, i only transfered 230 posts, so A LOT of links still point to the old forum. That should dissapear when the tool has processed more.
I’ve been taking a look at some of the really old stuff thats been appearing anyway, so it’s not really any hassle to cross-reference if anything looks looks bit wrong.
Is it better to let you know on discord or here so there’s a more long-term reference?
I can lend a hand with writing “about” sections for different sections. Probably won’t have the spare time for the cross referencing stuff you require mate
After all feedback has been processed ill delete nuke the entire archive category and everything in it and start over. Then all the feedback will show.
@MichaelX is the archive staff section open to public? Probably best it’s staff only like the current one. Staff organisation from 2007 is likely to be okay but especially as they get more recent we’d probably want them all locked down. Things like login sections and staff decisions about sensitive things may come up that we wouldn’t want to show up on Google haha
There are a few formatting errors based on syntax from the old board that no longer has function and therefore leaves artefacts.
The most important one I’ve seen that reoccurs is
 Â
Which, I think maybe is a tab or bullet point?
You also see a lot of html-style markup, [list] [/list] and the like. There’s also some that’s [] which must be Discourse interpreting [ * ] * (without the spaces)
Ill take those url’s that i saved in the database and do a lookup for the Discourse version of that page. If it’s found the post is updated and saved. The record is marked as complete.
,