I don’t normally like to create such inflammatory articles, but this time it’s absolutely warranted. Something is wrong with the Pokemon Go servers and the results are unexpected. Read on to find out what, or look at the image above closely for spoilers, or I suppose scroll past the boring problem statements and methodologies. There will be a lot of numbers and complicated concepts thrown around, but I will try and keep this understandable and entertaining.
Pokemon Go Raids have Bugs? Shocker!
Pokemon Go raid battles are not known for being the most bug free portion of the game. There is of course the well known start and end times of raid battles. Most often battles start at 297 seconds and end at 2 seconds eating up 5 precious seconds of time. Sometimes, especially with slow phones or bad internet, this 5 seconds can grow to 15 seconds, turning a close fight into a disappointing defeat. Dodge bugs in multi-player raids are also well known. Any attack that would normally kill you, if dodged, will put your phone into a buggy mode where only quick attacks are possible. Since most Pokemon get their damage from special attacks, this can also ruin attempts. There are other less well known issues that affect player success in raids, and Pokebattler takes great pride in investigating and updating its simulations to be as accurate as possible.
Pokebattler simulations are extremely accurate and well regarded at this point. There have been many tweaks to get the numbers as close to reality as possible, the most significant being December of 2017. The combat algorithm is well defined at this point. What remains are minute tweaks to adjust for unintended bugs in Pokemon Go raids. There are three major variables that can be tweaked to adjust numbers.
- Death Delay – This measures the amount of time in between the death of a Pokemon and the start of the first attack by the next Pokemon. This is used 5 times per battle party but is not used for the final Pokemon (as that falls under rejoin time).
- Quick Attack Lag – Lag is defined as unintentional delay caused by processing time, programmer bugs, internet delays and other side effects of being a multi-player game. Quick Attack Lag is the extra delay between the end of a quick attack and the start of the next attack. Quick attacks are queued on the server side, and generally have very short delays.
- Special Attack Lag – Special Attack Lag is very similar to Quick Attack Lag. It is defined as the delay between the end of a special attack and the beginning of the next quick attack. For unknown reasons, Special Attack Lag is higher than Quick Attack Lag.
These numbers were last updated in Version 5.0.1 From the patch notes:
- Reduced death delay from 2 seconds down to 1 second based on Game Master swapDurationMs value.
With this change, the numbers stood at:
- Death Delay: 1s
- Quick Attack Lag: 25ms
- Special Attack Lag: 75ms
For those of who really understand technology, these numbers should seem extremely suspicious. The death delay might be believable as a round number but having lag as an exact multiple of 25 milliseconds is pretty unbelievable. And you would be right. You’d be surprised how close it is though. Death delay though? That’s the big question. With ~8 deaths per player in a fight, that’s an extra 21 seconds time to win on a typical trio.
The most recent update was reported as too optimistic by famous Pokemon Go raider Kieng. You might known him from several world firsts including Lugia Trio and more recently a Ho-Oh Trio. If one of the best players in the world can’t match the times from the simulator, something is wrong.
What does Celandro do when his wife is out of town?
Celandro likes to look at frame by frame videos of Pokemon Go raids. What is a frame? Its 1/30th or 1/24th or some other fraction of a second of a video. If you pause a video, you see 1 frame. In Youtube you can move forward 1 frame by hitting the ‘.’ key and backwards with the ‘,’ key. I prefer to look at raw unedited mp4s in quicktime and use the arrow keys For hours on end. In fact 4 hours for 1 video in this case. No, I don’t know why I’m this obsessed over milliseconds, but I suppose someone has to be. I kept hoping someone else would do this research for me but almost as shockingly as Pokemon Go Raids having bugs, no one volunteered. So my wife heads out of town to visit her twin sister and I’ve got nothing better to do then to fill out spreadsheets with frame counts. Yes that was sarcasm.
Methodology 1 – Energy bar
Turns out the energy bar, for all its flaws, is a great way to count time between attacks. The energy bar moves for only two reasons: one you took damage, or two: you started an attack. A quick attack adds energy, a special attack removes energy. To determine how long between attacks, you simply count the number of frames in a video and divide by the number of frames per second. To find the lag between attacks, you take that attack delay and subtract the move duration. If there was no lag, the number would be 0. If there was good lag correction, the number would average to 0. And if that were true, you’d gain around 25 seconds on your time to win and I wouldn’t be spending hours on a Monday night trying to decide wether that was 33 frames or 34 frames. But I digress. There are issues with energy bars, and the main issue is energy rubber banding. This effect makes it look like you have energy and then you lose it and then you gain it back again and then you lose it. I’ll leave the theories for what is going on for the end, but let’s just say that this is annoying but bearable.
Methodology 2 – The Combat Log Never Lies
While using the energy bar for counting, I was having some extremely strange effects when attempting to determine the death lag. It turns out that the energy bar moves around sporadically immediately after death and then energy rubber banding occurs and things get quite confusing. Attempts to count the frames between a death and the next attack was confusing and not consistent between deaths at all. At some point in the hours of clicking the arrows on the keyboard, I noticed that the combat log on the right side has very interesting behavior. The log moves up to show ‘Super Effective’ exactly when the damage from a move should apply! So Wild Charge with a combat time of 600ms and a damage applied time of 300ms will gain energy, then get super effective in the combat log 300ms later then gain energy again 300 + lag milliseconds later. Using the time between when the color changed on the HP bar for death and the movement of the combat text gave very consistent results! Hurray! If anyone is reading this far and hasn’t skipped ahead to the pretty pictures I commend you, and trust me when I say I will get to the point.
Logs are funny things. They are simply lines of text that end up in a file somewhere and often scroll through ignored by everyone. Everyone but the accursed developer who needs to comb through them to try and figure out wtf is going on, that is. Logs are extremely important for debugging purposes and every developer worth their salt uses them specifically. What is a combat log good for a Pokemon Go Developer? Showing exactly what the server has sent the client! This is the source of truth. No one trusts the clients. Clients can be hacked. Server sends data. Put data in log. Trust the log. If the log is wrong, you have no way to debug a problem.
Counting Frames
A quick aside for anyone trying this at home. Counting frames is easier than it looks. I spent hours working out a repeatable metric for 'quickly' counting frames. The main techniques are muscle memory and memory tricks. And perhaps a lesson or two from the abacus. Counting to 106 consistently and quickly while moving a video forward frame by frame is easier said than done. Recounting is a huge problem and even a single frame off can throw off your numbers so you have to be extremely careful. The best way to do this would be multiple independent counters but noone wants to do this job remember? So here is how a single person does it.
- Count to 10 quickly in your head (unless noone is around), hitting the forward frame button one time for each number with your right hand.
- On hitting 10, say 10 loudly in your head and put one finger from your left hand on the desk.
- Repeat steps 1 and 2, placing an additional finger on the desk for each group of 10, resetting to zero fingers when you hit 50 or 100.
- Slow down when you near the expected frame
- When you see the frame you are looking for in the video, add up the number of fingers on your left hand + 10 times your current count
With this method you can count to 10 in about 3 seconds, or 100 in 30 seconds.
If you are imagining a dimly lit office and a crazy man tapping fingers on the desk for hours on end, taking occasional notes as he stares into the brightly lit screen, you've pretty much got the idea.
You are boring me, lag is normal in every online game. What’s this major bug?
Ok ok. Here’s the big bug: In certain situations, the attacker gets free quick and special attacks.
That’s right. The bank has made an error in your favor. You get extra damage. Far faster than you should. I don’t know what causes it. But when it does, the results are bizarre.
First Hydro Pump
Kyogre uses its first Hydro Pump
Unexplained Energy
A very large amount of energy is gained.
Second Hydro Pump
A second Hydro Pump shows up in the combat log for zero energy
Third Hydro Pump
A third Hydro Pump goes off along with 3 additional Waterfalls according to the combat log
Look at the combat log. Remember how I said combat logs never lie? There are 3 hydro pumps and 3 extra ‘Super Effective’ messages that must have come from Waterfall. And that’s not all! If you check the spreadsheet with the timings, tab ‘KyogreVsHoOh’, row 143, you will see that instead of one attack every 33-35ms as you would expect, you see two! The energy bar is going crazy in the video as is the combat log. Check out the left side attacker at 135 into this fight. The craziness doesn’t stop until the around 95s in. Thats basically 40s with extra Waterfalls and Hydro Pumps. I’m not sure this kill actually occurs without this bug!
So that’s the big news. No I don’t know what caused it. No I can’t help you solo ‘impossible’ bosses. I can speculate that a poorly synced phone clock might be involved. Or perhaps a rolling restart of a server. Then there is also the chance that despite my assurances about logs never lying, it’s possible that they do in this case. This may all just be a client server desync issue where the energy from the incoming special was applied twice. Except, I think not. This attempt was far better than every other attempt Kieng’s crew made. Perhaps they just got lucky you might say. It is theoretically winnable. And they did have quite a few cases of dying to the Brave Bird just after using Hydro Pump which is good DPS. But even so, I’m going to go with this being an actual bug. Because logs don’t lie. The server told the client that it did a whole bunch of extra attacks. It was confused about the energy which made everything bounce around but the moves happened.
But all I Care about is Simulation Accuracy!
The moment none of you have been waiting for: The results of my hours upon hours of counting frames till my wife comes home. 5 videos were analyzed. A Lugia trio kill, the Kyogre vs. Ho-Oh trio kill (x3) and a Rayquaza vs Ho-Oh trio failure.
- Death Delay – 1879ms average, 1909ms excluding best and worst 3. Turns out 2s was awefully close. Both the median and the mode was 1900ms and its very close to the average and the average excluding outliers. Pokebattler will be using 1900 ms going forward.
- Quick Attack Delay – 27.0ms average, 28.3ms excluding best and worst 3 from Rayquaza, and best and worst 5 from Kyogre (as well as the double attack section from Kyogre). Pokebattler will be using 28ms going forward.
- Special Attack Delay – 76ms average on non buggy Rayquaza fight (with ancient power spam). 66ms-68ms based on both Rayquaza and Kyogre. Pokebattler will be using 70ms going forward.
How is this different from a 2s death delay, a 25ms quick attack delay and a 75ms special attack delay? It’s not. Turns out those numbers were all awfully close. They might even be closer than the new numbers. I only looked at 5 videos. Then again, that’s 5 more videos than anyone else as far as I’m aware. Feel free to prove me wrong. Send spreadsheets and videos to [email protected]
Conclusion
The start and end times of a video are the most obvious sign that something is not quite right with the Niantic servers. Closer analysis shows an even bigger issue with server lag adding delays to every single attack. These delays add up to even further increase the time lost on boss attempts. Are these fixable? Sure. Simplest solution would be for Niantic to simply subtract 25ms quick attack delay and 75ms special attack delay from every move. While they are at it, increase the fight duration to 305 seconds. Is this correct? Close enough, at least to tide us over until whatever massive redesign they are working on. Is there a better solution? Of course. But this gives players the time they should have to defeat the boss.
In the meantime, Pokebattler will have the new delays with the next release expected later today with the game master update for Smack Down. That looks like a 16 power, 8-10 energy, 1100 – 1200 ms rock move that will make Tyranitar the undisputed rock champion. That will be especially great if more double super effective birds come around again.
I’d like to thank the people who helped make this post possible. Kiengiv, Broozes1876, Hund0Hunter, Cookedart and especially my lovely wife without whom this article would not be possible.
They have always had an issue where the client could send duplicate actions to the server. Often I get two candies walking a buddy. There was a glitch where when evolving that could get mon up to level 40 when the client bared it (now fixed).
Great article. It’ll be interesting to see how (or whether) Pokebattler’s recommendations change next time I use it.
BTW, thanks for mentioning that “,” and “.” trick when using YouTube. I’ve inched my way through a lot of how-to-catch videos (for Mewtwo, etc.) and it was painstaking because I didn’t know that trick.