AO3 News

Post Header

Published:
2020-08-17 09:53:14 -0400
Tags:

The last handful of releases included some shiny new features, such as the option to disable comments on works and the beginnings of a more robust system for managing Archive admins, as well as bug fixes and performance improvements.

We've also begun offering a Docker container image to make it even easier to contribute to the Archive's development. Thank you to tlee911, Tom Milligan, james_, Ariana, curtisxk38, elebow, and Z, who helped make this happen!

A special thank you to our first-time contributors, tlee911 and Tom Milligan!

Credits

  • Coders: Alix R, Cesy Avon, Elz, Enigel, james_, Jeff Kingswood (Littlelines), Matt Sears (Littlelines), redsummernight, Sarken, Stephen Burrows, Tal Hayon, teyla, ticking instant, tlee911, Tom Milligan, Tyme
  • Code reviewers: Ariana, curtisxk38, elebow, Enigel, james_, redsummernight, Sarken, ticking instant, Z
  • Testers: Adi M., Alison Watson, C. Ryan Smith, Chrome, Claudia, cosette, Ehryn, Hel, james_, Jessie, lydia-theda, Matty, Nary, Paula, QED, Qem, redsummernight, Relle, Runt, Sammie Louise, Sarken, ZoëRenee

Details

0.9.290

Bug fixes and performance improvements, deployed on June 28.

  • [AO3-5007] - It used to be technically possible to create non-canonical Warning, Category, and Rating tags. We have now added checks to prevent this. (See the 0.9.293 deploy for an exciting update to this fix!)
  • [AO3-5878] - If you edited your gift exchange sign-up using the "Edit Prompt" option and your new offer didn't actually meet the challenge requirements, your changes wouldn't be applied, but you also wouldn't know why. You now get a helpful error message when you try to submit your changes.
  • [AO3-5969] - The "Update Tag Filters" troubleshooting option (available to admins) wasn't working for all tags, and now it does!
  • [AO3-5737] - The blockquotes within userstuff (which contains work as well as FAQ content) always had a border on the right side, which looked off for right-to-left languages. It now applies the border on the left for those languages.
  • [AO3-5882] - The page title for the "Marked for Later" page was "Saved for Later", creating a certain tension between the heading on the page and the title displayed in the browser window or tab. This has been marked as resolved. (Boom-tish!)
  • [AO3-5907] - The "Share" button on works was preparing all the sharing options if you merely opened a work, creating unnecessary server load and needlessly saying hello to Twitter and Tumblr. Now it gathers up the work blurb (for easy copy-pasting) and creates the social media buttons only when pushed. We also improved user privacy by limiting HTTP requests to those third-party sites from work pages.
  • [AO3-5930] - The code that shows information about works within a series used to load all the works' fields from the database. It is now only fetching the fields it strictly needs, which is a great load off its mind, and less error prone when we change or rename some work attributes.
  • [AO3-5957] - We've updated the Rack gem to 2.2.3.
  • [AO3-5974] - We added the recommended workaround for a gem that was giving off security warnings, so it's not bothering us until we're ready for an upgrade.
  • [AO3-5976] - We made updates to our deploy script to account for new servers! \o/ (And decommissioned servers, RIP.)

0.9.291

Bug fixes and behind-the-scenes work, deployed on July 1.

  • [AO3-5771] - We made the links for tags revealed via "Show warnings" and "Show additional tags" go to the appropriate works page instead of the page for the tag.
  • [AO3-5881] - On the "Edit Multiple Works" page, we had checkboxes for the visibility options ("Only show to registered users" and "Show to all") which you could both check, destroying the space-time continuum in the process. It's now a proper set of radio buttons.
  • [AO3-5973] - We added a new column to the works table to store whether the work has comments disabled, in preparation for an option that lets you, well, disable comments on your works.
  • [AO3-5947] - We added a bunch of rules to the tool that hounds us about style issues in our code changes.

0.9.292

Better control over permissions for site administrators, deployed on July 29.

  • [AO3-5518] - We are introducing separate roles for site admins, with different permissions for Policy & Abuse, Support, Tag Wrangling, Translation, and other volunteers who might need access to the Archive's administration interface. To roll this out in a workable manner, we're defining actions and settings that we want to limit access to, so we can then give the relevant permissions to each admin role.
  • [AO3-5519] - Restricts the ability to create and edit AO3 News posts (like this one!).
  • [AO3-5883] - Restricts access to the Settings panel, where we configure things like how many invitations from our automated queue are sent each day.
  • [AO3-5884] - Restricts the ability to search for users and update user information, such as their Fannish Next of Kin.
  • [AO3-5885] - Restricts the ability to delete, hide, or modify user creations (works, series, bookmarks, and external works).
  • [AO3-5886] - Restricts the ability to delete comments or mark comments as spam.
  • [AO3-5980] - We updated the script we use to create admin accounts. Now it allows us to set roles and easily update admins' email addresses.
  • [AO3-5623] - Said script included a link that started with http:// and we updated it to the proper HTTPS protocol while we were at it.
  • [AO3-5979] - We were enforcing uniqueness constraints for admin accounts and their associated email addresses in the code, but not at the database level. Now it's airtight!
  • [AO3-5992] - We fixed a bug preventing our Policy & Abuse volunteers from periodically receiving reports on spam accounts.
  • [AO3-5967] - We fixed an issue where it was possible to use the "Import New Work" form on URLs that are otherwise only visible to the Archive's internal network.
  • [AO3-4923] - We added more automated tests for bookmarks.
  • [AO3-5999] - We made comments created in our automated tests better resemble valid comments on the actual Archive.
  • [AO3-5983] - We added another bunch of rules to the hounding tool.
  • [AO3-5793] - We now support Docker for creating a development environment! 🥳

0.9.293

Adding the option to disable comments on works and cleaning up faulty warning tags, deployed on August 14.

  • [AO3-5897] - You can now choose to turn off comments on your works.
  • [AO3-4275] - After preventing users from adding new Warning tags to the database (in the 0.9.290 deploy), we needed to clean up any existing tags that didn't belong in our very small set of Archive Warnings. This is now also done.
  • [AO3-4634] - It used to be possible for all logged in users to access the "Unposted Claims" page of a prompt meme challenge. Now this list of all claims is restricted to the challenge mods.
  • [AO3-5975] - We have added a banner that will be automatically displayed on proxy sites run by third parties, advising users that if they log in to their Archive account on the proxy site, the site owner can see their password.
  • [AO3-4925] - We added more tests and removed some unused code involving tag wrangling.
  • [AO3-6012] - We started using MariaDB instead of MySQL for our Docker and automated test databases.

Comment

Post Header

Published:
2020-06-24 13:39:57 -0400
Tags:

In this batch of fixes, we rewrote large parts of the code that updates work and bookmark listings when a tag is wrangled (e.g. made canonical or added as a synonym of another tag). Now the process is much less error-prone and we have some shiny new troubleshooting tools in case anything goes wrong. \o/ We also fixed several issues concerning hit counts and notification emails, improved our automated test suite, and generally tidied up a bit around the place.

Special thanks to new contributors J, Stephen Burrows, and Tyme!

Credits

  • Coders: Alix R, cosette, dlhardin, DreamSea, J, james_, redsummernight, Sarken, Stephen Burrows, teyla, ticking instant, Tyme
  • Code reviewers: bingeling, james_, redsummernight, Sarken, ticking instant
  • Testers: Alyndra, Booksarelife, C. Ryan Smith, Chelsea Eidbo, cosette, Danielle Strong, GoldenFalls, james_, lydia-theda, mumble, Paula, QED, Qem, Rebecca Sentance, redsummernight, Sammie Louise, Sarken, teyla, ticking instant

Details

0.9.286

Focus on hit counts and automated tests, deployed on April 22.

  • [AO3-4622] - The hit count preferences hadn't been working properly for many years, so we removed them.
  • [AO3-5905] - We've rewritten the hit count code so we can count hits while caching works. Check out our post Guest hit counting is coming back for more information on hit counts.
  • [AO3-4750] - We added a link to work blurbs that will take you to a work's latest chapter.
  • [AO3-5757] - We've updated the gem that runs our feature tests.
  • [AO3-5338] - In many of our automated tests, we use factories to create test data. The most frequently used work factory turned out to be creating drafts, not posted works, which wasn't great: in most cases, we want to know how our code interacts with the latter. We've updated the factory and the tests to run on works unless we specifically need to test drafts.
  • [AO3-5921] - Some of our automated tests would occasionally fail due to timing issues, so we made some changes to make them less flaky.

0.9.287

Focus on tag wrangling and filtering, deployed on May 28.

The always wonderful ticking instant -- who you may remember as the coder responsible for the massive speed improvements for gift exchange matching in 2016 -- rewrote some of the code responsible for making sure works and bookmarks are included in the proper tag listings.

The previous version of the code was subject to race conditions, so sometimes (e.g. during periods of heavy load) changes to the way a tag was wrangled wouldn't affect which listings a work (or bookmark) with that tag was included in. ticking instant has rewritten the code so it won't be subject to those race conditions and provided new troubleshooting tools in case something else goes wrong.

  • [AO3-5617], [AO3-4965], [AO3-3830] - Works would sometimes not appear in the tag listings after synning a tag to canonical, or making a tag into a canonical. Now they do.
  • [AO3-5618] - When an admin syns a canonical to another canonical (decanonizing the first canonical in the process), all of the connections (e.g. fandoms) from the formerly canonical tag are supposed to move to the new canonical. They were just vanishing, though. Now they'll transfer correctly.
  • [AO3-3768] - When a wrangler makes a tag into a synonym, the syn is supposed to pick up information from the canonical tag (e.g. a synned fandom tag should acquire the canonical fandom's media tag). It wasn't always doing that, but now it should be.
  • [AO3-4077] - If a wrangler removed a subtag from a metatag that had its own metatag, the works from the subtag would still stick around in the work listing for the metatag's metatag. Now the works will disappear, since the relationship between the two tags no longer exists.
  • [AO3-5343] - Wranglers would get an error when adding a metatag to a tag that had two levels of subtags. Now they can do that.
  • [AO3-5432] - Decanonizing a tag without first removing its metatags would sometimes fail to remove the formerly canonical tag's works or bookmarks from the metatag's listings. Now the works are removed.
  • [AO3-5433] - In certain cases, if an admin attempted to edit an external work to simultaneously add and remove a tag, the external work's bookmarks would stay on in the listing for the old tag and not appear in the listing for the new one. Now the bookmarks go where they're supposed to.
  • [AO3-5507] - Imagine, if you will, a tag tree structured thusly: a metatag (e.g. Animals) with the direct subtags A and B (e.g. Cats and Dogs), and then another subtag of subtag A (e.g. Ragdoll Cats). Now imagine posting a work tagged with both Dogs and Ragdoll Cats, making it appear in the Animals tag. But then, one day, a wrangling change comes! We remove Ragdoll Cats as a subtag of Cats. And like magic, our work vanishes from the listings for the Animals tag. Except we don't want magic here: our work is still tagged with Dogs, which is still connected to our Animals metatag. We've put an end to these sorts of disappearing acts, and made any works affected by this bug reappear in their proper place.
  • [AO3-5508] - Let's say we have the metatag Foods with the two subtags Fruits and Vegetables, and the tag Tomatoes is subtagged to both Fruits and Vegetables. Then, one day, we decide vegetables are not food (just kidding, veggies, we love you! 🥦), and we remove the Vegetables subtag from the Foods metatag. Furthermore, we decide tomatoes are not vegetables, and we remove the Vegetables metatag from the Tomatoes subtag. During all this wrangling, we were losing the connection between the Tomatoes tag and the Foods tag, which would keep new works with the Tomatoes tag from appearing in the Foods tag. Now we've made sure that connection stays intact, so anything posted with the Tomatoes tag will -- in our hypothetical scenario -- appear in the Foods tag as well (because Tomatoes are still Fruits). We've also fixed the problem on any existing works.
  • [AO3-5945] - Systems updated some settings to make sure users who were browsing the insecure version of the Archive wouldn't get bumped to the secure version.
  • [AO3-5783] - We removed an old rake task.
  • [AO3-5943] - We've told the tool that informs us about code style violations that there are some things we'd like it not to complain about.
  • [AO3-5927] - Some of our automated tests for the autocomplete would occasionally fail because the animation effect took too long to fire, meaning the autocomplete options weren't fully expanded by the time the tests started looking for them. We've disabled the animations to prevent these failures.

0.9.288

Focus on miscellaneous bug fixes and infrastructure updates, deployed on June 17.

  • [AO3-4874] - We were stripping HTML and paragraph breaks out of summaries in subscription emails, which could make them hard to read. We've stopped doing that.
  • [AO3-5332] - If a co-creator who was added only to one or more chapters (and not as co-creator of the whole work) removed themselves from the last chapter they co-created, they would still be co-creator for the whole work, with editing permissions. That clearly needed to be fixed, so we did.
  • [AO3-5721] - It wasn't possible to delete a challenge from a collection unless you had JavaScript enabled. We've made it work if you have JavaScript disabled.
  • [AO3-5938] - In the line of stats (words, comments, kudos, etc.) in a bookmark blurb, the links had lost their bottom border, making it impossible to tell that they were links. We've added some room for the borders, making the numbers pop again.
  • [AO3-5736] - When you create an account and try to log in before you have confirmed your email address, you now receive a more helpful error message, i.e. "Please check your email for the confirmation link."
  • [AO3-5266] - We now enforce HTTPS for embeds from Vidders.net.
  • [AO3-5940] - To prevent failures when sending out notifications, we are now making sure that all necessary information has been committed to the database (and not merely saved by the user, possibly needing a second to make its way into the system) before we put the email together and add it to the mailer queue.
  • [AO3-5965] - When a request comes in from a user with an expired session, we respond with an error, but we didn't do this for all possible request types. That caused errors on our end, especially due to the requests going to the hit counter. We're handling those in the proper manner now.
  • [AO3-5948] - All our cookies now last a maximum of one year.
  • [AO3-5828] - We have added the ability to safelist IP addresses so they won't be affected by restrictions (e.g. on the number of database requests per minute) that we impose through the Rack::Attack gem.
  • [AO3-5936] - On the actual Archive, we use a type of caching that we weren't using in our automated tests. Because we want our tests to resemble our actual set up as closely as possible, we've started using that sort of caching there as well.
  • [AO3-5960] - We updated the actionpack-page_caching gem to version 1.2.2.
  • [AO3-5971] - We updated the websocket-extensions gem to version 0.1.5.
  • [AO3-5422], [AO3-5423], [AO3-5425], [AO3-5426] - We removed unused code from a number of places. Yay, tidying!

Comment

Post Header

Published:
2020-04-14 15:39:56 -0400
Tags:

In recent releases, we wrapped up changes to the kudos table and began caching works for logged out users. We've also made improvements to subscriptions, spam-checking, the Statistics page, and addressed issues with the Terms of Service prompt.

A special thank you to our first-time contributor teyla!

Credits

  • Coders: bird, james_, redsummernight, Sarken, Tal Hayon, teyla, ticking instant, Z
  • Code reviewers: james_, redsummernight, Sarken, ticking instant
  • Testers: bingeling, coaldustcanary, GoldenFalls, Irina, james_, Lady Oscar, lydia-theda, MagnusIsMyRock, nimadge, Paula, Ranowa, Rebecca Sentance, redsummernight, Sammie Louise, Sarken, teyla

Details

Performance

  • [AO3-5162] - The site used to remember the name and email address that guests entered on the comment form, making it so they didn't have to reenter the information every time they left a comment. Unfortunately, we had to remove this code so we could begin caching work pages.
  • [AO3-5906] - As a side effect of our new caching measures, logged out users were being shown the adult content warning on every work rated Mature, Explicit, or Not Rated -- and when navigating between chapters of those works to boot. We've changed how we're setting the cookie that remembers you've already pressed the "Proceed" button on the warning and made a few tweaks to nginx, and now visitors should only see the adult content warning once per browser session.

Subscriptions

  • [AO3-5040] - You'll now get subscription emails for any anonymous works you've subscribed to. (But you won't get an email if a creator you've subscribed to posts an anonymous work -- that would ruin their anonymity.)
  • [AO3-5708] - A long time ago, we added some code to cancel the delivery of emails in certain situations, e.g. if you were subscribed to a creator who added their brand new work to an anonymous collection after posting, but before the subscription email went out. Rails 5 lets us handle situations like that without extra code, so we've gotten rid of it.

GDPR

  • [AO3-5497] - To comply with the GDPR, we present visitors with a prompt asking them to agree to the Terms of Service before using the site. When they agree to them, we set a cookie in their browser that is supposed to keep the prompt from reappearing for them. This wasn't working for some visitors, though, so we've decided to see if it works better if we switch to JavaScript's local storage feature.
  • [AO3-5915] - Our new approach to remembering whether visitors agreed to the Terms of Service didn't work in private mode in a few older browsers. So now we're checking if a visitor's browser supports local storage properly, and if not, using a cookie to remember whether we should show them the Terms of Service prompt.
  • [AO3-5917] - Some browsers didn't like our new approach to writing cookies or local storage items, particularly if you were in private mode, so we found yet another way to hide the TOS prompt once you've agreed to it.

Kudos

  • [AO3-5859] - Kudos used to belong to pseuds, but we're updating them so they belong to users instead. As part of this process, we've updated the kudos list on works to always list kudos-givers by their usernames, instead of sometimes displaying "pseud (username)".
  • [AO3-5887] - We've gotten rid of the column that stored pseud IDs in the kudos table and removed any code or tests related to it. We are so glad to be done with this migration!

Miscellany

  • [AO3-5193] - Wranglers used to be able to turn a canonical tag into a synonym in one fell swoop just by entering the new canonical tag in the "Synonym of" field on the tag edit page. Because it was too easy to do this accidentally, we've made it so only admins have access to that field on canonical tags' edit pages.
  • [AO3-5821] - We've made a couple tweaks to the graph on users' Statistics pages: the vertical axis will never start at a number other than zero, and now the bars on the graph are a slightly faded Archive red rather than bright Google blue.
  • [AO3-5908] - Visiting a multi-chapter work on the insecure site would redirect you to the secure site, which was terribly inconvenient if you were using the insecure site because you can't access the secure site. (It's the only reason you should use it.) We've made it so you'll stay on the insecure site.
  • [AO3-4539] - Our spam blocker occasionally prevents legitimate support tickets and abuse reports from being submitted. Because reports and tickets from logged in users are generally not created by spam bots, we've stopped running the spam check for them.

Behind-the-scenes miscellany

  • [AO3-5895] - Our friendly gem-updating bot updated our JSON gem to version 2.3.0.
  • [AO3-5898] - There was a possible security issue with the way Rails handles some JavaScript code, which would normally be something we could resolve with a quick gem update. However, we're on Rails 5.1, and fixing it with a gem update was only an option for sites on Rails 5.2 or 6. Instead, we've added a monkey patch to hold us over until we update Rails. 🐒
  • [AO3-5893] - Every time our automated tests run, the tool we use to create PDF downloads gets downloaded and installed on our testing environment, allowing us to test downloads. The source for the tool moved to a new location, so we updated the test setup to reflect that.
  • [AO3-5896] - Because we run our tests a lot, we would occasionally get rate limited by GitHub and be unable to download our PDF generation software. We've moved it to our own servers so we won't have that problem.
  • [AO3-5892] - The randomly-generated text in our email tests sometimes includes characters that are not in the US-ASCII character set, such as ç. Whenever that happened, our email tests would fail because they were looking at the encoded email text. We've told the tests to check the decoded text from now on so they'll pass no matter what characters they contain.
  • [AO3-5911] - We've updated our automated tests to use version 6.8.8 of Elasticsearch so we can make sure everything works with it before we actually update the version running on our servers.
  • [AO3-5904] - Some of our automated tests related to orphaning would occasionally fail because that's a fun thing tests do sometimes if they're run too quickly. We added slight delays to them so they'll stop doing that.
  • [AO3-5900] - Our automated tests were actually submitting things to our spam checker, and since we do this repeatedly and frequently, the spam checker thought they were spam. This caused our tests to fail. We've started stubbing the results of the spam check so that doesn't happen.
  • [AO3-5899], [AO3-5902], [AO3-5914], [AO3-5916] - We've kept our deploy script up-to-date: removing two servers we've converted into firewalls, then three decommissioned servers, and adding a second front end server to our staging environment so our caching set up there more closely resembles our set up on production.

Comment

Post Header

Published:
2020-03-18 19:16:03 -0400
Tags:

Our latest releases include improvements for new users, site admins, creators of anonymous or unrevealed works, and the kudos database table.

A special thank you to our first-time contributor Morrgani!

Credits

  • Coders: cosette, james_, Morrgani, redsummernight, Sarken, Tal Hayon, ticking instant
  • Code reviewers: Ariana, james_, redsummernight, Sarken, ticking instant
  • Testers: Arcadion, C. Ryan Smith, Capi, Danielle Strong, GoldenFalls, hel, james_, lydia-theda, MagnusIsMyRock, Matty, Paula, redsummernight, Sammie Louise, Sarken

Details

User Accounts

  • [AO3-5723] - We've tweaked the text of the email you receive when you create a new account. It will now be more consistent with our usual way of wording things.
  • [AO3-5316] - When you first activate a new AO3 account, you're supposed to get a success message. That message wasn't working, which led users to follow the activation a second time, which also didn't display a message it was supposed to display. Both of those messages should turn up now, hopefully leading to less confusion and fewer Support tickets.

Admin

  • [AO3-5676] - The script that we use to handle GDPR data requests from users used to write to a file, which was slightly annoying to get to. We've made it write to standard output, a.k.a. the screen that the admin running the script is looking at.
  • [AO3-5863] - We also updated the GDPR data requests script to handle some changes we made when we moved the logging in process to Devise.
  • [AO3-3547] - Our Policy & Abuse team has always been able to add notes to user accounts when giving warnings or suspensions, but sometimes they wanted to make a note without doing either of those things. We've given them that ability.
  • [AO3-4205] - To help with spam and abuse, we let admins see the IP address of the user who posted a work. However, we forgot to record that information on works imported using the "Post without previewing" option, so there was nothing for admins to see. We've now corrected that oversight.

Kudos

  • [AO3-5851] - In a previous release, we added a column to the kudos table to store the user (rather than the pseud) that left kudos. Now we've gone back and filled in this information for existing kudos.
  • [AO3-5869] - We have around 640 million kudos on the Archive, so the database table we keep them in was getting rather full. We moved them to a larger table that will be able to hold 9,223,372,036,854,775,807 kudos (yes, really!). The new table also has a uniqueness constraint that was missing from the old one, so now any duplicate kudos that sneak past our code will also be rejected by the database.
  • [AO3-5868] - When a logged in user leaves kudos on a work, we used to check to make sure the user's default pseud hadn't previously left kudos on the work. Now we check that the user hasn't left kudos on that work before.
  • [AO3-5873] - When we moved all the kudos to their new table, some works experienced a change in their kudos counts due to duplicate kudos being removed. We've made sure the kudos counts for those works are updated in our search engine.

Testing

  • [AO3-5874] - A few months ago, the hardware for our staging environment went belly up, so we began temporarily using a cloud service. While we were using this temporary setup, our heroic sysadmin james_ went and got the broken hardware from its facility, repaired it, drove it back to the facility, and plugged it back in. Shortly afterwards, we updated our configuration to put the newly repaired hardware back into service.
  • [AO3-5875] - In January, we started using a gem to password protect our staging site. We've switched to a different form of password protection and removed that gem.
  • [AO3-5872] - We've reorganized our automated tests to match the structure RSpec automatically assumes we're using. This will help make sure all of our tests are actually being run.

Miscellaneous

  • [AO3-5857] - We restored the default Dutch translations that got lost when we updated our translation tool.
  • [AO3-4139] - In Reversi, we've changed the color of the little footnotes on required fields to make them more readable. Goodbye, dark grey text on a slightly less dark grey background!
  • [AO3-4836] - In the unlikely scenario you were trying to approve all the moderated comments on a work you didn't own, you'd get a 500 error. Now you'll get redirected to the homepage with an error message.
  • [AO3-5028] - If you add your work to a collection and the moderators later make the work anonymous or unrevealed, you'll now receive an email letting you know about that change and reminding you how to remove your work from the collection if you're not okay with the change. (Please note that you'll get this email even if you have your preferences set to disallow emails from collections.)
  • [AO3-5879] - We -- or more accurately, the friendly bot who lets us know about security issues with gems -- upgraded the version of the nokogiri gem we're using.
  • [AO3-5807] - There was a column in the support ticket database table that we weren't using and thus wanted to get rid of. This worked out quite well because it let us test a method of modifying database tables that we wanted to use for our work on the kudos table.
  • [AO3-5888] - We got the new Elasticsearch servers we've been waiting for! 🎉 This allowed us to repurpose some old servers, which required us to update our deploy script.

Comment

Post Header

Published:
2020-02-10 08:59:20 -0500
Tags:

In recent releases, our top priority has been to reduce Elasticsearch-related slowness. Along the way, we've fixed an eclectic selection of bugs related to skins, kudos, orphaning, embeds, and internationalization.

A special thank you to our new contributors, bird, Jen Rogers, and Z! Also to derenrich and ticking instant, who spotted two security issues!

Credits

  • Coders: Alix R, Ariana, bird, cosette, cyrilcee, derenrich, Elz, Enigel, james_, Jen Rogers, Nerine Luna Cyran, redsummernight, Sarken, Tal Hayon, ticking instant, Z
  • Code reviewers: Ariana, bingeling, cosette, james_, redsummernight, Sarken, ticking instant
  • Testers: C. Ryan Smith, captured_in_lspace, CJ Record, cosette, Frost The Fox, GoldenFalls, Lady Oscar, lydia-theda, MagnusIsMyRock, mumble, Nary, nimadge, james_, Rebecca Sentance, redsummernight, Sarken

Details

Performance

  • [AO3-5832] - We added a new error page regarding our Elasticsearch-related slowness. (But our heroic sysadmins did a lot of work to make sure very few people would ever have the opportunity to be on the receiving end of that error!)
  • [AO3-5834] - We've made it so we can adjust the cache expiration in the work and bookmark controllers without a code deploy.
  • [AO3-5823] - To help lighten the load on our servers, we added rate limiting for searches (and pages powered by Elasticsearch, e.g. tag works pages).
  • [AO3-5841] - When we can't figure out the IP address of someone accessing the site, we use the site's internal IP address. Unfortunately, that meant some folks were getting rate limited when they shouldn't have been. We've made sure that won't happen.
  • [AO3-5848] - Rack::Attack -- our rate limiting tool -- does things in a certain order no matter what order the code is in. It's easy to be unaware of that, though, so we reordered our code to convey that to future coders.

Skins

  • [AO3-5184] - In certain versions of Safari, it wasn't possible to access the Advanced section of the skins form because the button toggling that section was missing. We've found it and put it back.
  • [AO3-4030] - For tag wranglers using the Reversi skin, some of the information on tag edit pages was unreadable. We've changed the text color to make it readable.
  • [AO3-4241] - In Reversi, it was difficult to read text you entered in the header search box if the box was currently selected. We've made it easier to read by changing the color.
  • [AO3-5796] - We've added two new media query options for skins: (prefers-color-scheme: dark) and (prefers-color-scheme: light). If your device supports it, you can use these options to create skins that are only applied when your device is set to dark or light mode.

Internationalization

  • [AO3-5806] - A long long time ago, we set up the tool our Translation team will be using to translate the Archive into other languages. This configuration no longer worked quite right, so we set it up again.
  • [AO3-5813] - Once we got our translation tool set up, we realized some of our email code needed tweaking to make the emails easier to translate... so we did that tweaking.
  • [AO3-5735] - We now automatically add HTML direction attributes to Arabic and Hebrew news posts, making text correctly flow right to left. Our translators no longer need to set them manually on every single post.

Orphaning

  • [AO3-5046] - If you wanted to orphan all the works by one of your pseuds, but one of those works happened to have more than one of your pseuds on it, one of two un-good things would happen, depending on when you encountered the bug: either you'd get a 500 error, or both of your pseuds would be removed from the work. We've fixed it so it doesn't error and only removes the pseud you specify.
  • [AO3-5854] - Previously, we didn't check that you owned the thing you were orphaning, so if you were clever and bad, you could've conceivably orphaned someone else's works. We check that now, so this should no longer be possible.

Embeds

  • [AO3-5818] - We previously changed our referrer policy to limit information other sites could get if you followed a link there from a page on the Archive. Unfortunately, this caused some YouTube embeds to stop working, so we had to change the policy to something slightly more permissive. (If you're interested in the specifics, we're now using strict-origin-when-cross-origin instead of the previous same-origin.)
  • [AO3-5853] - It was possible to upload random HTML files to archive.org and embed them in works, so we have to specifically whitelist their media player.

Kudos

  • [AO3-5844] - We added a new column to the kudos table that will allow us to store the user ID (rather than the default pseud ID) of the person who left kudos.
  • [AO3-5850] - We began storing user IDs in the aforementioned column when users leave new kudos.
  • [AO3-4934] - We increased test coverage of the kudos code (it's at 100% now, yay!) and fixed a bug that would occur in the unlikely event you had JavaScript disabled and were trying to leave kudos on a work the creator had deleted.

Buttons and text changes

  • [AO3-4991] - We've made things a bit more consistent by rearranging the buttons on series pages to better match the way buttons are ordered on work pages.
  • [AO3-5624] - Also on the "consistency is good" front, we made the order of the Reply, Delete, and Thread buttons on comments on the homepage match the order of the buttons in the comments section.
  • [AO3-5845] - Sometimes, particularly in Firefox and Chrome, the Twitter share button didn't show up on a work's or bookmark's share modal. We've made it more likely to show up.
  • [AO3-5738] - When you first create an AO3 account, you're taken to a page with some important information. We've updated the text to make that information a bit clearer.
  • [AO3-5113] - The error message you got when entering a different password and password confirmation (e.g. when creating an account) was a bit wrong. We've made it right.

Miscellany

  • [AO3-4194] - On the page for requesting invitations for friends, some of the form fields weren't quite up to accessibility standards. Now they are! (Unfortunately, you'll have to take our word for this: we had to turn off access to this page due to spammers.)
  • [AO3-5773] - If you bookmarked an external work with an ampersand (&) in the title, it would display as &. We've made it appear as a regular old & now.
  • [AO3-5362] - We've tweaked the position of the background image on the left side of the login page so it overlaps a bit less with the text on the right side.
  • [AO3-3847] - If you renamed a series, the old name would stick around on the work header and blurb. Now the new name will show up, as one would reasonably expect it to.
  • [AO3-5233] - When admins searched for users by multiple emails, the number of results would be off if users have emails with different capitalization. We fixed such numbers.

Behind-the-scenes miscellany

  • [AO3-5759] - We've added password protection to our staging site. It's better if search engines cannot see our works in the "Testing" fandom that just repeat the word "test" 500 times.
  • [AO3-5762] - We sped up our tests by only setting up search indexes for tests that use them.
  • [AO3-5817] - Automated tests for emails would fail if the emails included certain characters, e.g. ç. We've fixed it so the tests will pass.
  • [AO3-5838] - We improved automated test coverage of the bookmarks page because our testing volunteers found a bug our tests didn't pick up. (This is why manual testing is important!)
  • [AO3-5839] - We increased test coverage for pseuds.
  • [AO3-5840] - We updated the Rack gem to version 2.0.8.
  • [AO3-5836], [AO3-5847], [AO3-5849] - We shuffled some servers around, so we updated our deploy script to make sure new code goes to the right place.

Comment

Post Header

Published:
2019-12-21 07:16:00 -0500
Tags:

The last several code deploys introduced some exciting changes to work posting: a big update to our rich text editor, support for the HTML5 audio and video tags, a little tweak in how we handle language selection, and some less confusingly labeled buttons! In addition, a whole lot of challenge improvements and behind-the-scenes infrastructure upgrades.

Credits

  • Coders: Alix R, cosette, Elz, Enigel, james_, redsummernight, Sarken, Tal, tickinginstant
  • Code reviewers: Ariana, bingeling, Enigel, james_, redsummernight, Sarken, Scott, tickinginstant
  • Testers: bingeling, Briar, CJ Record, GoldenFalls, Izzy, james_, Lady Oscar, lydia-theda, MagnusIsMyRock, McBangle, Nary, nimadge, Paula, Rebecca Sentance, redsummernight, Relle, Sarken

Details

Rich Text Editor

  • [AO3-5191] - We upgraded the version of TinyMCE that we use for our Rich Text Editor.
  • [AO3-5809], [AO3-5811] - The new version of TinyMCE overrides the browser's default context menu, which prevents desktop users from right-clicking to copy, paste, spell check, etc. We disabled this behavior.
  • [AO3-5812] - The same new version of TinyMCE defaults to a reduced interface for users on mobile devices. We disabled this behavior too.

Collections and Challenges

  • [AO3-4353] - Some inconvenient caching was affecting the display of tag set content (e.g. Character or Relationship tags) on challenge sign-up pages. Now the correct tags are shown.
  • [AO3-5786] - Trying to access the list of allowed tags from a challenge sign-up would throw an error 500, which is now fixed as well!
  • [AO3-5785] - Owners of collections were previously able to add works in other, anonymous collections to their own, which enabled them to see the work's creator. We've now made it impossible to add anonymous works to your collection.
  • [AO3-5101] - Trying to save a challenge with sign-ups open and a closing date set, but no opening date, would throw an error 500. Now the challenge profile correctly states that you can sign-up and until what date.
  • [AO3-5102] - If you closed sign-ups for a challenge before the set closing date, the challenge profile would show that sign-ups were closed, but ~in the future~~~ (i.e. the set closing date). Now the date just isn't displayed anymore, to avoid any confusion.
  • [AO3-5593] - When fetching a list of all collections a work had been approved to be in, we used an unnecessary query, which was not good for database performance. Much better now!
  • [AO3-4571] - You have two open assignments, and they both happen to be for the same recipient. You post a work fulfilling one of the assignments. However, the other assignment's ticky box is also checked in the posting form. If you don't notice this, you post the work and suddenly your other assignment is fulfilled as well. But it's really not! (Aww.) We now make sure only the correct ticky is ticked and you still have to write a second work for the other assignment. *\o/*

Language

  • [AO3-5702] - We previously had English set as the default language when posting a new work. This led to a lot of miscategorized works, since creators often wouldn't notice the language selection when posting non-English works. We now require all users to manually set the language their work is in.
  • [AO3-4605] - The form used for work imports now sports a mandatory "Choose a language" field as well.
  • [AO3-5790] - We accidentally broke filtering for works in português brasileiro or português europeu, so we intentionally fixed it.
  • [AO3-5616] - In our lists of FAQ categories available in Arabic or Hebrew, the first category name was misaligned. We've put it back in line.

Misc.

  • [AO3-5693] - Posting audio or video fanworks? Now you can use the HTML5 audio and video tags!
  • [AO3-4080] - We had a very peculiar bug where our devs weren't able to use multiple CSS box shadows on their development environments despite it being possible on the Archive itself. Turns out we had a broken regular expression in our code, but the fact that we had box-shadow in the wrong part of the production configuration file was hiding the issue. The bug has been fixed and the production config file updated, so now box-shadow works the same for everyone.
  • [AO3-4560] - We renamed the buttons for posting works: "Post Without Preview" becomes "Post", and "Save Without Posting" becomes "Save As Draft".
  • [AO3-5632] - When you follow a link from the Archive to another site or go to a page with the Twitter widget (e.g. a work), some information is automatically sent to the site (or Twitter). We have now limited referrer information and won't tell the site you clicked on (or Twitter) which page you were on.
  • [AO3-5761] - We have added the Rack::Attack gem, which enables us to control how many requests to the database an IP address can send before we firmly tell it to not (and serve an error page instead).
  • [AO3-5639] - Trying to visit an unrevealed work (and getting a helpful message instead) or accessing a hidden work (as an admin) would increase that work's hit count, potentially giving the creator of said work quite the scare. We have removed that particular source of panic now.
  • [AO3-3617] - Tags that begin with a capital letter with an accent or other diacritical mark (e.g. Éomer) wouldn't show up in the autocomplete if no fandom was entered in the posting form. Now they should!
  • [AO3-5825] - Dashboard sidebars retrieve work and bookmark counts from Elasticsearch, which ties up Elasticsearch in long queries, making everyone sad. We are now caching those counts, creating a little less work for the search engine every time a page with the sidebar is loaded. \o/ (Note: This means the count will no longer go up immediately when you add or delete a work or bookmark. The delay should currently be about half an hour, but we may adjust it in the future.)

Infrastructure

  • [AO3-5789] - Updated Database Cleaner and other test-only gems.
  • [AO3-5795] - Updated MySQL2 to 0.5.2 to support libmariadb and Debian 10.
  • [AO3-5798] - Updated Loofah from 2.2.3 to 2.3.1.
  • [AO3-5749] - Updated Elasticsearch to 6.8.5 and its gem to 6.8.0.
  • [AO3-5826] - Updated Nokogiri to 1.10.5.
  • [AO3-5797], [AO3-5800] - Updated the staging server names in our deploy script.
  • [AO3-5194] - Updated schema and structure files.
  • [AO3-5787] - We've put the code for the public site skins Reversi, Snow Blue, and Low Vision Default in our code repository for safe keeping and better tracking of changes.

Tests

  • [AO3-5788] - We had previously been using Ubuntu Xenial for tests on our CI service, but now we've switched to Bionic, which is closer to our production environment.
  • [AO3-5814] - Sometimes we make changes that we don't need our automated tests to check. We've made it so we can include a note in the commit message that will tell one of our CI services to skip running the tests and deploy the changes straight to our staging environment.
  • [AO3-5815] - The automated tests for our emails had a lot of redundancy, so we reduced that.
  • [AO3-5758] - We changed some settings on our automated code style checker to make its formatting suggestions more useful.
  • [AO3-5772] - We've stopped running automated code style checks on JavaScript files for third-party plugins, since we didn't write them and aren't going to address style violations found in them. We've also stopped checking minified JavaScript files because, seriously, we know it's one giant long line without spaces or indentation -- that's the whole point.

Comment

Post Header

Published:
2019-10-21 14:14:04 -0400
Tags:

In our latest series of deploys, we made several improvements to searching and filtering, cleaned up a whole bunch of broken links, and made some finicky, but necessary changes under the hood to upgrade Ruby to the latest version.

An extra special thank you to our first-time contributor dlhardin!

Credits

  • Coders: Ariana, dlhardin, Elz, Enigel, james_, lethnie, Matt Sears (Littlelines), MaxwellsDaemon, redsummernight, Sarken, Tal Hayon, ticking instant
  • Code reviewers: Ariana, Elz, Enigel, james_, redsummernight, Sarken, ticking instant
  • Testers: Alison Watson, Ariana, Briar, briar_pipe, Claire P. Baker, cosette, GoldenFalls, Izzy, james_, Katherine, Matty, Memé, mumble, Naomi, Rebecca Sentance, redsummernight, Relle, Sammie Louise, Sarken, Taylor Clossin

Details

Searching & Filtering

  • [AO3-5492] - Searches using tags with numbers in them (e.g. "Episode S01E03") would completely ignore the numbers (and return all works with Episode tags, for example). It now respects your choices.
  • [AO3-5707] - When we renamed the class for Archive warnings (more details about that in the Infrastructure section), we realised this could break searches made with the old code (such as bookmarked searches, or searches in your browser history) by allowing them to include unwanted results. We ensured that those searches will keep working as intended.
  • [AO3-5649] - Sorting works by creator names would sometimes lead to confusing results for co-created works, since a work "by Alec, Ellie" could show up under E. It now follows the actual alphabet and sorts by the first creator listed.
  • [AO3-5652] - You can now search for works by the title of the series a work is part of. \o/
  • [AO3-5703] - Works and bookmarks are now searchable by their language codes, i.e. two- or three-letter abbreviations that are easier to look up than the IDs our database assigns to each language. You can read more about this in our hidden search operators cheatsheet.

Broken linkage

  • [AO3-5583] - When accessing the comment section on a work or news post via a direct link, the pagination links would jump you to the top of the page, away from the comments. We've now made sure the anchor stays firmly in place.
  • [AO3-5179] - We used to have an older page with information about orphaning, to be read before orphaning a work. We now point to the Orphaning FAQ instead, which is more neatly maintained.
  • [AO3-5688] - However, when deleting an account, the "What do you want to do with your works?" page did not link to any information on orphaning, leaving users to wonder what that option would do. Now it links to the Orphaning FAQ as well.
  • [AO3-5076] - The Terms of Service FAQ was using old URLs to link to specific Archive FAQ items, leading to errors. Those links have now been fixed.
  • [AO3-5077] - Also in the Terms of Service FAQ, some links to subsections of the Terms of Service were broken. We fixed those, too.

Misc. Fixes

  • [AO3-5673] - The fandom, user, and work counts on the Archive homepage used to be hard to parse, so we added commas as thousands separators. Much neater now!
  • [AO3-5730] - On the posting form, the labels for chapter number and title weren't properly associated with the respective form fields. Now clicking on those labels will select the field so you can enter the information.
  • [AO3-5733] - The assignments page used to show an extraneous email link when a user had multiple assignments in the same exchange. We have fixed that.
  • [AO3-5024] - Trying to log in as an admin with incorrect information would just quietly fail, leaving the admin puzzled why nothing was happening. It now displays a helpful error message.

Infrastructure

  • [AO3-5613] - Previously, our code used a class named Warning for Archive warning tags. Newer versions of Ruby, however, wanted that class name for themselves, so we had to rename ours to ArchiveWarning before we could upgrade the version of Ruby we're using. (We used warning tags in more places than we ever imagined...)
  • [AO3-5756] - We updated the Devise gem to version 4.7.1.
  • [AO3-5655] - One of our test files ended up in the wrong folder where it was never run. We moved it to the right place.
  • [AO3-5750] - We updated a library used by the testing suite -- the gem formerly known as factory_girl is now factory_bot.
  • [AO3-5391] - We removed unused code related to tags.
  • [AO3-5784] - We've upgraded the version of Ruby that the Archive runs on to 2.6.5 (the latest stable version at the time of coding).

Comment

Post Header

Published:
2019-09-07 03:30:43 -0400
Tags:

The new creator invitation system and related preference are the main changes in this release, but because they required a significant overhaul of the creator code, we also fixed a number of related bugs in the process!

Credits

  • Coders: james_, Nerine Luna Cyran, Sarken, ticking instant
  • Code reviewers: redsummernight, Sarken, ticking instant
  • Testers: Ania, Ariana, bingeling, doughtier, james_, memorizingthedigitsofpi, Matty, mumble, Nary, redsummernight, Sammie Louise, Sarken, Stephanie Godden, ticking instant

Details

Adding creators and co-creators

  • [AO3-5658] - We overhauled the way co-creating functions by adding an invitation system and requiring users to opt-in before they can be invited. For more details, please check out our post, "Important changes to adding work, chapter, and series co-creators."
  • [AO3-4998] - If two or more users had the pseud Mary Sue, it wasn't possible to add either of them as a chapter co-creator unless you just happened to write their name in a very specific format. We've made it so it'll work without knowing that format. Specifically, if you enter a pseud that belongs to multiple users, you will get an error asking you to choose the proper user from a dropdown menu.
  • [AO3-5606] - We fixed the same issue with series, too!
  • [AO3-5662] - If you were trying to add both a co-creator and a series to a work at the same time, but there was an error adding the co-creator due to that whole "two Mary Sues" scenario we described above, the series would vanish and not get added to the work once you clarified the co-creator situation. We did a little work to make sure the series gets added now, too.
  • [AO3-5061] - If you pressed "Preview" when making changes to your work's or chapter's creators, the proper creators weren't listed on the preview page. Now they are! (This does not apply to creators who are only invited -- they won't show up on the preview.)
  • [AO3-5620] - It used to be that adding a co-creator to a chapter wouldn't add that co-creator to any series the work was in. Now it does!
  • [AO3-5663] - Previously, if you had a series and added a co-created work to that series, the other user(s) on the work would not get listed as co-creators of the series. Now they will.
  • [AO3-5261] - When someone added you as a co-creator, the email used to only say, "You've been added as a co-creator." Now we tell you exactly who added (or invited) you, which can be helpful if there's more than one other person on the work or series.
  • [AO3-3915] - Using the form for editing multiple works to add a co-creator to multi-chapter works didn't add the person to all of the works' chapters. Now it does!

Removing creators

  • [AO3-5334] - The option to remove yourself as a work or series co-creator previous said "Remove Me As Author," which makes certain assumptions about what you're editing. We've changed it to say "Co-Creator" instead. (We also tweaked the resultant error or success message to refer to creators rather than authors.)
  • [AO3-5083] - While using the newly renamed "Remove Me As Co-Creator" button on a series correctly removed you as one of the series' creators, preventing you from making future edits, it incorrectly left your name on the series. Now your name will vanish along with your editing privileges.
  • [AO3-5692] - On the form for editing multiple works, there was an option for removing co-creators. This option didn't work and wasn't actually something we wanted to provide, so we removed it.

Miscellaneous

  • [AO3-5475] - The mass edit form used to give a rather unhelpful 500 error if one of your changes was invalid (e.g. you were trying to add a tag containing a banned character like * or ^). Now it will take you back to the form with an error message that tells you what the problem is and provides you with an opportunity to fix said problem.
  • [AO3-5625] - Thanks to a bug we believe was generally caused by changing a work's creators using the mass edit form, it was occasionally impossible to edit a work you'd created. (Basically, while transferring the work, the bit in the database responsible for saying, "This user owns this chapter of this work," would go missing.) We've made it so the bug shouldn't happen anymore and so any works already affected by the bug should be editable now.
  • [AO3-5659] - Once upon a time, we planned to make a /browse page for every user, and we had some code relating to it in place. This was so long ago that we'd kind of forgotten why we'd ever wanted that page, though, so we deleted the code.
  • [AO3-5751] - Admins were previously only able to see draft works if they were one chapter and if they were given a direct link, which made it difficult to provide support for draft-related issues. Now admins can see drafts with multiple chapters and draft chapters on posted works. They can also access a user's Drafts page in case the user doesn't provide a direct link.
  • [AO3-5753] - The character limit for comments was previously set to 4,300 characters -- a nod to the limit on LiveJournal, where many AO3 coders and users came from in the early days. But that was a long time ago, and some people like to leave much longer comments, so we've increased the limit to 10,000 characters.

Infrastructure

  • [AO3-5729] - We upgraded to a more recent version of the nokogiri gem.

Comment


Pages Navigation