Internet Rot and brain drain: The loss of technical knowledge as sites and forums decay

Gungriffon Geona

Hi there.

I made my account back when I was very young and frankly I hate this stupid username. But I don't feel like making a new account to post what I feel is an important message. I hope you will read what I have to say.

       I work as a pixel artist and 3D artist, and pixel art has had such a gigantic brain-drain that people are still trying to reverse engineer effects created by the masters of the arcade era. For me to learn in the 2000s like I did, I spent majority of my time tearing apart ripped spritesheets and taking screenshots to learn. And it guaranteed was a much slower method than it should have been. 

       There has been significant work in recent years to solve this issue, but even now we have situations of artists simply not sharing their findings, leaving the amateurs to amateur. Still, it's atleast getting better, slowly but surely.

Which leads me to coming back here, and the reason I write this:

       A friend of mine is trying desperately to learn 3D in his college courses, and I am often flabbergasted by the lunacy his "totally worked at pixar" professor spews, and how little he's actually been taught about doing 3D in a professional setting, or outright told the wrong thing to do.

So, I did what I usually like to do: link back over to this site. the place I learned from all those years ago.

Only... There's a problem:

       In my hunt to find old tutorials and information on normalmaps, hard surface, subdivion, etc, I've noticed that a huge slew of images are gone. Whenever a thread directs to the polycount wiki, multiple examples and tutorials are simply gone: lost to the sands of shitty site hosting, photobucket, and other such rot that has made it harder to find things that are actually useful. 

       In it's wake, we have youtubers and new students who will never see this information. An algorithm that filters out small helpful videos with specific knowledge in favor of garbage like "The Donut tutorial". "Professors" who's understanding of 3D is out of date by decades or who seemingly teach wrong on purpose. Amateurs and trolls teaching amateurs how to amateur. A trend I'm growing constantly frustrated by.

       So what exactly is being done to archive this information before it's lost for good? A non-insignificant number of the resources linked in the "Subdivision Surface Modelling" article alone are dead. Who is making sure this information can be learned outside of the field? 

       This is a serious issue that needs to be resolved, and I would like to hope someone here is trying to backup the work done to teach and help others. If not, steps need to be taken to make things functional, even if it's just dumping things on archive.org or somewhere less unstable.

Literally anything to get people to stop thinking the bevel modifier in Blender is a remotely viable method for doing hard-surface modelling. Please, I beg of you.

Eric Chadwick

It's unfortunately a never-ending battle. Internet addresses are short-lived.

However, we're dedicated here at Polycount to providing a place to store content and keep it up indefinitely. Anything uploaded directly to Polycount will be preserved (drag and drop, people!), as well as anything attached directly in the wiki.

If you post something here and you want to remove it, just Edit your post. Or, Flag/Report the post to catch our attention and we'll help out.

If you're willing to help hunt things down and find them, we're happy to give you a wiki editing account. We can also change usernames, wink wink. 

il Duce Primo

Gonna chime in say that other forums come and go while polycount still going strong.

Gungriffon Geona

Eric Chadwick said:

It's unfortunately a never-ending battle. Internet addresses are short-lived.

However, we're dedicated here at Polycount to providing a place to store content and keep it up indefinitely. Anything uploaded directly to Polycount will be preserved (drag and drop, people!), as well as anything attached directly in the wiki.

If you post something here and you want to remove it, just Edit your post. Or, Flag/Report the post to catch our attention and we'll help out.

If you're willing to help hunt things down and find them, we're happy to give you a wiki editing account. We can also change usernames, wink wink. 

I have been very much considering making my own videos about the more obscure side of modelling for awhile now, but I suppose the big thing has been finding the best method to explain things. Part of me is hoping someone else here may have archived info from some of the threads or websites that are gone/images missing, which is part of why I made the thread.

I would like to help however I can, but it does also require help from those that have been here more consistently than me, for lack of a better phrasing.

Alemja

I reached out to Eric to possibly help out with the wiki. I also put out some feelers to a few other people who have been around for a while to see if anyone has saved information that is older but now gone from the internet, like maybe old competition stuff from Dominance Wars

Turns out I do have PDFs and images of the old Subdivision Modeling thread for their discussion on the Pole. I do have some old SDKs that used to be hosted on Game Artisans or other forums that are now gone, and I also recently made a backup PDF of the Allods Online English translation that used be hosted by 80.lv, but the subdomain is gone.

I am working on getting the Subdivision threads in a more readable format so they can be put on the wiki, but I can dump the source files if people are interested and want them

dimwalker

Eric Chadwick said:

However, we're dedicated here at Polycount to providing a place to store content and keep it up indefinitely.

Custom quake 3 character models are gone =(

poopipe

dimwalker said:

Eric Chadwick said:

However, we're dedicated here at Polycount to providing a place to store content and keep it up indefinitely.

Custom quake 3 character models are gone =(

 I'm not sure i ever want to see my contributions to that again

sevsev

Another thing that bothers me immensely is the way search engines have deteriorated over the years. I have a harder time finding what I am looking for than when I started out with 3D almost a decade ago. Not only because content is disappearing, but because the algorithms can't seem to find it anymore. I search for things I know exist, but then they take a lot of dedication to actually find. There is a lot of misinformation as well, which makes it especially hard if you're just starting out and you don't even know what you should be looking out for. 
There are so many tutorials I wish I had downloaded over the years, because so many have been taken down or the websites they were on don't even exist anymore. I didn't think they would disappear that quickly and now they are gone. 

aprats91

sevsev said:

Another thing that bothers me immensely is the way search engines have deteriorated over the years. I have a harder time finding what I am looking for than when I started out with 3D almost a decade ago. Not only because content is disappearing, but because the algorithms can't seem to find it anymore. I search for things I know exist, but then they take a lot of dedication to actually find. There is a lot of misinformation as well, which makes it especially hard if you're just starting out and you don't even know what you should be looking out for. 
There are so many tutorials I wish I had downloaded over the years, because so many have been taken down or the websites they were on don't even exist anymore. I didn't think they would disappear that quickly and now they are gone. 

Sometimes https://duckduckgo.com/ is a good option for searching without being so at the mercy of algorithms. It would be interesting to create together a reliable source of information and have control over the content on servers that are under our control and that would have the option to download the entire source in a compressed file to have the information offline if necessary.

Melomad

God you are more than right about a lot of things. I draw and do 3D, aiming to get a job with that. I'm always on the lookout for things to watch and learn in my down-time, and the videos that are recommended to me are mostly garbage ; Especially the 2D art related ones. Now there are good teachers and ressources out there, but the amount of shit I have to shovel through to find them is astounding. There is plenty of stuff for beginers and almost nothing for intermediates. Thankfully traditional art techniques, skills and knowledges are applicable to the modern digital artist. I can read a book and learn from it ; something I've found to be incredibly difficult with 3D. Softwares and techniques move fast and it's hard to keep up. I've learned a ton from just talking w/ people working in games, but it's not something everyone has access to. Missing pictures in the wiki are annoying, but I'm glad it exists at all!

myclay

Alemja said:

I reached out to Eric to possibly help out with the wiki. I also put out some feelers to a few other people who have been around for a while to see if anyone has saved information that is older but now gone from the internet, like maybe old competition stuff from Dominance Wars

Turns out I do have PDFs and images of the old Subdivision Modeling thread for their discussion on the Pole.

I am working on getting the Subdivision threads in a more readable format so they can be put on the wiki, but I can dump the source files if people are interested and want them

yeah dump the files so we can fill the holes.
Once there is one person sharing, others will follow.
Here are some zips of the subdivisionmodelling forum (the pole zip is too big for the forum)

Eric Chadwick

We’re sorting out an issue with the smtp forwarder for the wiki at the moment, but once that’s fixed we can grant new wiki editing accounts.

pior

By the way, one suggestion (which is probably not as simple to implement as it seems, but perhaps could be done as a community effort if needed) would be to provide a downloadable version of the Wiki as PDF - either as individual pages or as a whole. That way anyone could have a backup, or even rehost if needed (if allowed of course).

iam717

Yarp my go to was .pdfing everything.

Use alternative engines (more than 3) for them old goldies but i am pretty sure 'they' are locking down the ability to "help one's self", be seeking the information out, (the digital walls going up with the writing on them.) More for the youth to get "trapped" in these new things than any of the wiser folks.

i might have some of those Q3A models, not 100% sure though, been a long while, the one that came to mind immediately was the camper/tent model, it rings true to this day in gaming, the camper lives on, it just changed forms!

thomasp

Don't forget that beyond web tutorials there are also several (at least two?) of those Vertex e-books from a few years ago. I recall those being full of little niche-tutorial gems.

Its terrible that forums are going out of fashion and are being replaced by reddit and perhaps Facebook and Twitter feeds. That stuff is so hard to search and/or annoying to use and as a result you end up seeing only newbie posts asking the same beginner questions every day, flushing out whatever interesting content there may have been from last week. The subreddits on game art/Unreal and the like are a sorry sight.

Anyway I too find search engines getting worse - Google at the very least. Not only are the results flooded with SEO garbage but I read on one of their blogs that they even toned down the effect of using search engine operators (include, exclude, search for exact string matches) over the years.

Duckduckgo seems better - I made it my default on all devices.

Internet around 2010 was best Internet. Change my mind!

Alemja

The vertex books are still available on Gumroad, but it would probably be great to have copies on the wiki too:

https://ryanhawkins.gumroad.com/

Trying to attach the PDFs and images for the Subdivision forums "The Pole" and "The Pole Revised" hopefully the forum is okay with the size of them

Alemja

Here is the English Allods Online Stylized 3D guide. This one I actually took the time to transcript from the archive.org version back in the fall and updated some links that had decayed. Some of the images in the PDF might be smaller than their source, so I'm throwing those up in a separate zip too.

This one I have a source word processing file of the transcript and would be extremely easy to throw onto the wiki. It's waaaay to big for the forum though, almost double the allowed size.

myclay

Here are the siggraph (2004) files from Jeff Unay regarding Modeling According to Muscle Flow and Triangle cleanup.

okidoki

Well.. it's simple.. often images were stored "elsewhere"  and sometimes software migrations work not so  good as expected .. and nowadays it's sometimes more like a: look what i have made.. more on..
Weirdly enough you can find a lot of interesting images from PolyCount on  Pinterest and when clicking the link.. then the images are broken.. so you do not know exactly which belongs to which post.. and also can't just post them into this threads because of unclear copyright or if the author is okay with (when you decipher this relation )

Even some of the added link will fade away in some time

 ( In fact i'm searching some old tut's for the (even do not know the correct term) glossy/specular workflow (?).. i mean "before PBR".. but that's another story... )

Gungriffon Geona

     Thank you all for sharing things so far! I've been on a bit of a vacation right now, but I'll look over everything as soon as I can! I can already tell these will be a big help for me teaching my friends or organizing thoughts for making videos. Especially glad that I can see where things were originally from, so citing things back will be easier that way.

     I'm actually most excited about the Stylization 3D Production Guide, as that has always been one of the harder things for me to get across in my more wibbly-wobbly way of explaining things that aren't as concrete.

Hopefully other folks have things to share and archive! Thank you so much for responding!

dimwalker said:

Eric Chadwick said:

However, we're dedicated here at Polycount to providing a place to store content and keep it up indefinitely.

Custom quake 3 character models are gone =(

Ah, man. I actually miss alot of that stuff, though I was more of an Unreal Tournament/UT2k4 sorta person. The Freedom Force community has also done such a bad job at archiving creations that some things I've asked them about, no one had ever heard of ten years ago. It makes me sad, because I used to love looking them over when I was a teenager and wanting to know how to do that, and now I want to see them again just to see how far I've come by comparison, haha.

Polynaught

Regarding useful posts on polycount, anyone know what happened to Frank Polygon? Looks like his account is gone.

Eric Chadwick

Thanks everyone for the great archives, will be great to get those up on the wiki. 

I am also currently on holiday (first time hiking Moab, amazing place!) so I don’t have access to a proper pc, to process everything properly.


If Vertex is available on Gumroad then it should stay there, so people can donate to Ryan for all his hard work, if they choose to. Offering copies elsewhere, like on the wiki, should ideally only be done as a last resort.

Frank Polygon in December asked for his account to be locked out, but for the content in the Sketchbook (https://polycount.com/discussion/221392/sketchbook-frank-polygon/p1 ) to remain visible. He has an open invitation to reopen the account whenever he’s ready.

On archiving/restoring broken images, I spent a bit of time finding all the original images from this thread (Pirate castle (UDK)) by searching Wayback Machine, reverse image searching, etc. It was worth it IMHO, but it’s a fair amount of effort. People get tired of this after a while, understandably. That’s why the wiki falls out of date after a bit; there’s only so much time at the end of the day. But I do still applaud efforts to help archive and spread knowledge.

Eric Chadwick

Oh and wiki PDFs, you’re welcome to download whatever. We don’t have tools for this, but I’m sure you can figure it out There are even other wikis out there which have copied our content wholesale, which is ok AFAIK. As long as the point is to share info and not profit off it, I think we’re ok with that. Same idea as with the Greentooth.

Gungriffon Geona

I've been away for a bit due to my old man having alot of health issues, but looking through my files I have a texture pack that was specifically meant for Quake 3 Arena mapping. I've been messing around alot with doing low-res models and environments, so I think these will be of interest for anyone else, if not simply preservation's sake.

https://mega.nz/file/e05GmYpA#_W3Lo0JPGj1QkDLQbn-zG5-KeEWzICIQJS-JY_sBNfI

hunhow

Odd question, but: How would one even go about archiving the information? Screenshots with saved images and files? What's the best method?

myclay

I would suggest reading thru a couple official archive file type recommendations from different archives and countries.

Like this one; 

https://www.archives.gov/records-mgmt/policy/transfer-guidance-tables.html

Screenshots of full pages (which I made a couple years ago) worked imo better than saved out htm, html files since the interpretation of the files within browsers can drastically change thru time.

Context of what is where placed in a page can be quite important for understanding the content.

PDFs can be used depending on the container type. Avoid pdfs which are not standard e.g. with embedded videos.

Avoid filetypes which are not publicly and fully documented and worse if they are not easily readable by humans.
As an example, png is to be preferred over closed and secret file formats like .psd , .afphoto etc.

file formats like fbx are abysmal for archival purposes. use gltf instead.
Ideally you also archive programs (portable versions) alongside the data for easy access.

Source code of those programs is also great.

Saving the contents additionally in separate formats helps to further increase the recoverability.
Think about the job of a future archive maintainer, do they need OCR just to copy out the text out of an image?
Try to make it easier and provide the text alongside.

images can also rot in digital format, different files change differently thru time (dealing with partial data loss) and recoverability attempts need to be taken into consideration.

Do the future archive maintainers need special hardware which is soon EOL? Avoid that, move everything to new more available hardware.

An Archive is only as much worth as it gets actively maintained, translated and updated to new circumstances.

Some hardware features are to new for the archived software and the archive members have to find a passable solution.( modifying, recompiling software| transforming the data while keeping it as close to the original source as possible)

Keeping the originals in good condition is important as well as updated editions.

okidoki

Choose your weapons...

https://github.com/ArchiveBox/ArchiveBox/wiki/Web-Archiving-Community