403 Fobiddenn Error

[Maetrim of Cannith]

Hi Wiki controllers,

I am trying to update the DDOBuilder with the U80 items. To do this I run the command "Wget.exe "wiki link", but I am getting a 403 forbidden error when trying to download the files.

Has something changed to prevent a bot crawl of the wiki? If something has, is there a variation to the command line I can do to get this to work, as going back to creating all the items manually it going to be a PITA.

Mae

 

[ChickenMobile]

Hi Wiki controllers,

I am trying to update the DDOBuilder with the U80 items. To do this I run the command "Wget.exe "wiki link", but I am getting a 403 forbidden error when trying to download the files.

Has something changed to prevent a bot crawl of the wiki? If something has, is there a variation to the command line I can do to get this to work, as going back to creating all the items manually it going to be a PITA.

Mae

Are you an Australian, using Vodaphone network?

Edit: and yes bot crawling has been limited. AI absolutely slams the wiki.

 

[Maetrim of Cannith]

Are you an Australian, using Vodaphone network?

Edit: and yes bot crawling has been limited. AI absolutely slams the wiki.

I am in the UK. I will be investigating further to see if i can use my wiki login while doing the crawl. This crawl only happens once on a new update and has to be triggered manually by me and the code tries to limit it only to the item page files linked in the source url.

 

[Maetrim of Cannith]

As a note, in the short term I have manually downloaded each item file. I could not figure out how to get wget to use my login wiki credentials :(

 

[ChickenMobile]

Update:
Xevo is working on a solution that would allow legit crawlers in the future but still blocks spam traffic.
If you want to discuss other options join us in the wiki discord.