Forum:Plan to set up a troping wiki, need some questions answered

Forums: Index &rarr; General discussion


 * Hi Geth. Here are my answers to your three questions, respectively:
 * We can upload large text dumps and images directly to the server. We can enable uploads on your wiki, although we recommend the use of Wikimedia Commons for everything that isn't copyrighted or fair-use.
 * Sure.
 * We are constantly upgrading our infrastructure as new donations come in and we think we should be able to handle your wiki's traffic without any problems.
 * Kudu (talk) 16:15, 20 October 2013 (UTC)


 * Thank you for the answers Kudu. It will take some time before I am ready to create a wiki as we are still gathering the material for import, but I had another question: Since Orain supports custom extension development, would they also support improving extensions that need updated functionality?


 * To be specific, when/if I am ready to start this wiki, I would like to use a fully functional version of the following extension:


 * https://www.mediawiki.org/wiki/Extension:LinkSuggest


 * From what I understand, it's has not been updated, and while it works on MW 1.20, it does not work as well as the version currently deployed on Wikia, which has been updated further with Wikia specific code, whereas the original version of this extension has essentially ceased to be updated. I would greatly like to use a version as functional as the Wikia version on my prospective wiki, and I was wondering I could request Orain modify the extension to have better functionality, as while its not necessary for my prospective wiki, it would be incredibly useful. GethN7 (talk) 22:43, 20 October 2013 (UTC)GethN7


 * Hi Geth, no problem, you can take your time to gather the material. As for the LinkSuggest extension, we're currently busy with some other technical tasks since we're a fairly new site, so we won't be able to write any new custom code of our own for that extension, although we would be happy to install it if someone from your community wants to update it. Kudu (talk) 22:07, 21 October 2013 (UTC)


 * Thanks again for your response, Kudu. When we have the content ready for import, I will be ready to request a wiki for creation. As for the extension, I understand and wish your staff well with the tasks they have at hand, and if I can be of any assistance, do let me know. In the meantime, thanks for answering my questions. GethN7 (talk) 23:00, 21 October 2013 (UTC)GethN7

Hi, I'm Vorticity, and I'm in charge of getting the import data for this site ready to go. This means converting a mountain of data from a highly customized PmWiki install to MediaWiki markup. I'm parsing roughly 165,000 pages, averaging something like 120kB per page -- this works out to ~2GB of textual data. GethN7 is my entire QA crew for this, but at least import is working for him. Anyway, since all of the pages are assembled from individual files, what size would you like the chunks of XML import data to be?

As for the image set, mentioned above: The problem is that our data source, TV Tropes, is very loosey-goosey about licensing. The images are marked, along with the rest of the site, as CC-By-SA, but the subject of the site is literary criticism of copyrighted media after all. So the vast majority of the images are likely fair use -- and there's no real way to tell a priori which is which. Not without going through 50k images by hand. The good news here is that they're low-resolution. The entire image database is also about 2GB, the same size as the textual data. (After I wrangled about 15% more compression out of them with pngcrush and jpegtran, that is.) The current plan is to tag as "CC-By-SA 3.0 or Fair Use, please edit this description if you know which applies, or send DMCA requests to foo@whatever". Either way, we'll have to start with hosting with you, then move the files to the commons that we can get away with moving.

Anyway, I plan to be involved in running this thing too, so feel free to contact me as well.

Oh, also I can't create an account here. I get this after herding the cat images: This is a cached copy of the requested page, and may not be up to date. Sorry! This site is experiencing technical difficulties. Try waiting a few minutes and reloading. (Cannot contact the database server) --2602:304:CD9B:2950:3615:9EFF:FE08:CB5E 07:59, 23 October 2013 (UTC) (vorticity)


 * Hi Vorticity, your problem with creating an account today may have been related to [//botbot.me/freenode/orain/msg/7151797/ this logged IRC discussion], when there was a slight hiccup with the Orain wikifarm database today, but it looks like it is all sorted out now. You should be able to create an account now. Please let us know if you are still experiencing problems with this. Augur NZ &#x2710; &#x2315; 10:37, 23 October 2013 (UTC)
 * Thanks. Account creation finally worked, though I was still getting errors up through Noon PST (maybe a bad cache?).  Anyway, I would like an answer on this:  What would be the best size for my Import XML chunks to be?  A 2.2GB file of everything (no one's using FAT32, right?), 50 files at ~50MB, one file per page?  As soon as I get an answer, I'll be able to generate the entire data set.  Thanks Vorticity (talk) 00:18, 24 October 2013 (UTC)
 * Please inform me as soon as possible as well. GethN7 (talk) 13:12, 24 October 2013 (UTC)GethN7