Ensure character birth century

[jkisor]

Issue: #193

Root cause

Modern uploads from emulators appear to be uploading some birth dates with no century. For example, 01/01/20 is uploaded instead of 01/01/2020. Treestats.net expects format %m/%d/%Y seen here

Date.strptime returns a year of 0020 which reproduces the issue.

require 'date'
Date.strptime("01/01/20", "%m/%d/%Y")
# => #<Date: 0020-01-01 ((1728363j,0s,0n),+0s,2299161j)>

Solutions

There's three parts to the solution:

• Write: Add century to dates as new records are uploaded
• Read: Add century to dates before rendering (html and json) existing records.
• Cleanup: Find all Character's with missing centuries and update the records.

NOTE: I've solved all three, but you can pair the "Write" and "Cleanup" solutions while omitting the "Read" solution if you like. This will make the diff much smaller.

Points of interest (main branch)

Here's all the locations in the code for reading and writing. Let me know if there I've missed any.

For writing (1) :

• treestats.net/routes/upload.rb

  Lines 69 to 79 in 4dc8a0f

  	if(json_text.has_key?("birth"))
  	json_text["birth"] = CharacterHelper::parse_birth(json_text["birth"])
  	end
  	# Log extra debug info if birth ends up being nil
  	if json_text["birth"].nil? && ENV['RACK_ENV'] != 'test'
  	puts "Failed to parse birth field with the following JSON..."
  	puts json_text
  	end
  	character = Character.unscoped.find_or_create_by(name: name, server: server)

For reading (4):

• Account page:

  treestats.net/helpers/account_helper.rb

  Line 42 in 4dc8a0f

  :value => Proc.new { |v| v[:birth] }

• Rankings page:

  treestats.net/helpers/rankings_helper.rb

  Line 466 in 4dc8a0f

  :accessor => Proc.new { |v| v["b"] }

• Character page (Other pane):

  treestats.net/views/_other_pane.haml

  Line 7 in 4dc8a0f

  %td= @character.birth

• Character JSON output:

  treestats.net/routes/server.rb

  Line 65 in 4dc8a0f

  JSON.pretty_generate(@character.serializable_hash({}).tap {|h| h.delete("id")})

Cleanup

Run bundle exec rake cleanup_birth
Sample output:

Found 1
Updating...
Done

The task can be re-run to ensure it finds 0.

[jkisor]

The only 19xx date is 1999, which was retail -- It seems to have been consistently with a century.

Any uploaded birth without a century going forward is assumed to be 20xx. This means this will be incorrect during the year 3000+. Even if Asheron's Call and Treestats.net live that long, I can confidently say we won't. 😉

[jkisor]

This method is unused.

[amoeba]

Thanks for taking the initiative on this @jkisor. I will try to take a look at this in the first part of this upcoming week.

As a heads up, I think my preferred plan here will be to do a better job parsing squirrely birth values at upload-time and assuming any birth values stored in the db are correct and can be used/displayed without modification. Makes things simpler all around. And I'd want to follow that up with some general cleanup like nulling birth values like 0020-03-26 20:35:30 UTC.

[amoeba]

Sorry for taking longer than I had hoped to look over this @jkisor. As with your other recent PRs, thank you for the diligence in describing the issue and providing solutions and rationale. It helps me dive in more quickly than I otherwise would.

I took a look at the database and noticed basically two types of bad values for birth:

1. 1970-01-01...
2. Stuff like 0020 (like you saw)

I'm not sure what causes the first, but it's clear that some code is catching a null which gets turned into the UNIX epoch datetime. Maybe this has been fixed in emulator updates. For (2), what's really going on is that I screwed up a long time ago before I knew any better and handled dates in a non-culture-invariant way: amoeba/treestats#34. You may be familiar, but it simplifies everyone's life if the plugin just handles dates invariant of culture (locale), I didn't do that, and now we have bad data.

Since it's only a few records out of the whole and it's ultimately an issue I can fix from the plugin side, I'm inclined to avoid changes to the site and would rather fix the values directly in the database and fix the plugin. I went ahead and did that in 1c3f242 and things look good again. I effectively followed your Cleanup strategy, just with a slightly modified script.

It does look like active players are the cause of some of the odd values for birth so this might crop up again. I'm going to keep an eye on things and work on getting a new version of the plugin out with amoeba/treestats#34.

Thanks for taking such an interest and working up these detailed PRs. It really does help the projects, even if I don't merge everything.