Please upgrade your browser for the best possible experience.

Chrome Firefox Internet Explorer
×

Guild Ship missing

First BioWare Post First BioWare Post

Nee-Elder's Avatar


Nee-Elder
10.19.2020 , 04:47 PM | #191
From tomorrow's big UPDATE and Patch Notes: http://www.swtor.com/patchnotes

"Fixed an error which could cause guild strongholds and flagships to be unavailable."

Rejoice!
Never tell me the odds!

Kaveat's Avatar


Kaveat
10.19.2020 , 09:59 PM | #192
Quote: Originally Posted by Completely Fake and Unofficial Source
We are happy to announce that we have discovered that the "missing" guild ships are not actually missing, but were hijacked by ewoks in a galactic effort to return home before the next ewok holiday special starts production.

Unfortunately, as Endor is not yet in the production schedule for general access, we are unable to access said guild ships to facilitate their return, but will be addressing this issue in a future game update.
fixt, Happy Patch Day!
[end of line]
I have never 'suffered' from insanity... I always enjoy every minute of it!

JenHarkness's Avatar


JenHarkness
10.22.2020 , 10:59 AM | #193 This is the last staff post in this thread.  
Hi everyone! I know the guild ships have been disappearing for a lot of guilds lately, and it seems like it's been around forever. The good news is that things should be much better after the release of 6.1.4. Since this has been around a while, I wanted to take a bit of time to give you all a peek behind the curtain to help explain why it was so hard to track down. I also really want you to understand just how much we appreciate your forum posts and bug reports. Even when we don't reply, as a developer my first stop in investigating a backend issue is always looking at logs, and the more detailed the reports are, the easier it is for me to find something.

A bit of background on the Backend
Like many games, we store our persistent character data in a database. To make updates faster, we have a cache layer between the game and the database that batches up updates. Then we write the batch to the database. The benefit is we get better performance, but the drawback is that if there is one bad update in the batch, we abort the entire batch. Most of the time, the effect of aborting is that characters get kicked to character select, which isn't great, but is better than the data being corrupted. The good thing is, these errors are pretty rare, and are always the highest priority for us to fix.

So why might an update fail? Well, one of the nifty things you can do with a database is define a constraint on your data. That's a rule that can never be broken. Now the database is your last line of defense against something going wrong and your data getting corrupted. For this story, the important constraint is on unique character names. If we try to write a batch of updates, and one of the characters in that update has the same name as someone else in the database, the batch gets aborted.

A Victim of our own Success
Now, you all might remember some complaints on the forums lately about character renames not working. Behind the scenes, we have a big list of all the names on each server, so that if someone tries to use a name that's taken, the rename gets rejected before it gets anywhere near the database. Well... how does that list of names get built, you might ask. Basically, before we let you guys into the servers after a restart, the servers ask the database for that list of names so that they can make quick decisions later.

Here's the part where we have become a victim of our own success. It turns out that the programmer who wrote that database query back in 2009 or so added a parameter for how far back in the past to search for taken names. He happened to put 3000 for the default number of days. Yup, that's a bit over 8 years, and we launched a bit less than 9 years ago. It's a bit off because of some database maintenance we did the first year, but you get the point. So anyone who tried to rename a character with one of the names that was last played 8+ years ago would have the rename look like it succeeds, but then the database says no.

Luckily, that was a pretty easy fix, just increase the number. (Although if we start having rename issues in 2093, feel free to yell at me!)

OK.. but why would this have anything to do with guild ships??
Well, it turns out that sometimes the data that stores info on your guild ships was part of the batch that was being kicked out because of a bad rename. Yes, I know the guild ship problems have been happening for years, but that's because anytime we were having database errors, sometimes the guild ships were being caught in the crossfire. However, like I mentioned above, we always fixed the database errors super quick, so people would complain about guild ships, but then we couldn't reproduce it and they died down, because we'd already fixed the database error.

Here's the good news, this time it's really fixed. We've both fixed the rename error, and thanks to the guilds who have given super timely info, we were able to spot the guild ships getting caught in the crossfire. Now that we know what's going on, we were able to make it so that if the guild ship does get batched up with a bad batch, it will be automatically reloaded. So, no more disappearing flagships even if we have database*errors in the future. For realz this time.* (We hope!)
Jen Harkness | Senior Server Programmer
Follow us on Twitter @SWTOR | Like us on Facebook
[Contact Us] [Rules of Conduct] [F.A.Q.]

Nee-Elder's Avatar


Nee-Elder
10.22.2020 , 11:49 AM | #194
Quote: Originally Posted by JenHarkness View Post
Hi everyone! I know the guild ships have been disappearing for a lot of guilds lately, and it seems like it's been around forever. The good news is that things should be much better after the release of 6.1.4. Since this has been around a while, I wanted to take a bit of time to give you all a peek behind the curtain to help explain why it was so hard to track down. I also really want you to understand just how much we appreciate your forum posts and bug reports. Even when we don't reply, as a developer my first stop in investigating a backend issue is always looking at logs, and the more detailed the reports are, the easier it is for me to find something.

A bit of background on the Backend
Like many games, we store our persistent character data in a database. To make updates faster, we have a cache layer between the game and the database that batches up updates. Then we write the batch to the database. The benefit is we get better performance, but the drawback is that if there is one bad update in the batch, we abort the entire batch. Most of the time, the effect of aborting is that characters get kicked to character select, which isn't great, but is better than the data being corrupted. The good thing is, these errors are pretty rare, and are always the highest priority for us to fix.

So why might an update fail? Well, one of the nifty things you can do with a database is define a constraint on your data. That's a rule that can never be broken. Now the database is your last line of defense against something going wrong and your data getting corrupted. For this story, the important constraint is on unique character names. If we try to write a batch of updates, and one of the characters in that update has the same name as someone else in the database, the batch gets aborted.

A Victim of our own Success
Now, you all might remember some complaints on the forums lately about character renames not working. Behind the scenes, we have a big list of all the names on each server, so that if someone tries to use a name that's taken, the rename gets rejected before it gets anywhere near the database. Well... how does that list of names get built, you might ask. Basically, before we let you guys into the servers after a restart, the servers ask the database for that list of names so that they can make quick decisions later.

Here's the part where we have become a victim of our own success. It turns out that the programmer who wrote that database query back in 2009 or so added a parameter for how far back in the past to search for taken names. He happened to put 3000 for the default number of days. Yup, that's a bit over 8 years, and we launched a bit less than 9 years ago. It's a bit off because of some database maintenance we did the first year, but you get the point. So anyone who tried to rename a character with one of the names that was last played 8+ years ago would have the rename look like it succeeds, but then the database says no.

Luckily, that was a pretty easy fix, just increase the number. (Although if we start having rename issues in 2093, feel free to yell at me!)

OK.. but why would this have anything to do with guild ships??
Well, it turns out that sometimes the data that stores info on your guild ships was part of the batch that was being kicked out because of a bad rename. Yes, I know the guild ship problems have been happening for years, but that's because anytime we were having database errors, sometimes the guild ships were being caught in the crossfire. However, like I mentioned above, we always fixed the database errors super quick, so people would complain about guild ships, but then we couldn't reproduce it and they died down, because we'd already fixed the database error.

Here's the good news, this time it's really fixed. We've both fixed the rename error, and thanks to the guilds who have given super timely info, we were able to spot the guild ships getting caught in the crossfire. Now that we know what's going on, we were able to make it so that if the guild ship does get batched up with a bad batch, it will be automatically reloaded. So, no more disappearing flagships even if we have database*errors in the future. For realz this time.* (We hope!)
Post of the Year imo ^

As a 2011 SWTOR 'founder' (and former QA-Staff for a certain notable emulator of a game that rhymes with Bar Boars Falacies ) who not only "critizes" BioWare from time to time but also recently reported/posted on this very particular issue, words cannot express how much i appreciate that ^^ level of communication and detail.

Obviously, devs & coders are typically super busy and rarely have time to partake in these forums, but still...i really wish EA would allow more BioWare devs to post from behind-the-code-curtain like this more often. imho, it would go a long way with building (or re-building) community trust as we head into what's probably gonna be the twilight of SWTOR's 10+ year run here soon.

So, THANK YOU Ms. Jen Harkness, hectic hardworking dev who hasn't posted since 2013 (and thank you Mr. Musco for endorsing the transparency) .

./tip-hat
Never tell me the odds!

TUXs's Avatar


TUXs
10.22.2020 , 12:41 PM | #195
As this has recently happened to our guild on 2 occasions in less than a month, I have to give a HUGE THANK YOU for figuring out the issue. I know your work can be thankless at times, but your description and explanation as to why it was happening is very very much appreciated!

Again...THANK YOU!!!
All warfare is based on deception If his forces are united, separate them If you are far from the enemy, make him believe you are near A leader leads by example not by force
My referral code: here What you get: here (1 FREE transfer 7-day FREE sub FREE Jumpstart and Preferred Bundles)

Hudlar's Avatar


Hudlar
10.22.2020 , 12:55 PM | #196
Thanks so much for the update! I really enjoyed the peek behind the scenes.

UlaVii's Avatar


UlaVii
10.22.2020 , 03:00 PM | #197
Thank you for helping all our guilds and for the great post explaining the issue and solution

HKtheindomitable's Avatar


HKtheindomitable
10.22.2020 , 03:52 PM | #198
Quote: Originally Posted by Nee-Elder View Post
Post of the Year imo ^

As a 2011 SWTOR 'founder' (and former QA-Staff for a certain notable emulator of a game that rhymes with Bar Boars Falacies ) who not only "critizes" BioWare from time to time but also recently reported/posted on this very particular issue, words cannot express how much i appreciate that ^^ level of communication and detail.

Obviously, devs & coders are typically super busy and rarely have time to partake in these forums, but still...i really wish EA would allow more BioWare devs to post from behind-the-code-curtain like this more often. imho, it would go a long way with building (or re-building) community trust as we head into what's probably gonna be the twilight of SWTOR's 10+ year run here soon.

So, THANK YOU Ms. Jen Harkness, hectic hardworking dev who hasn't posted since 2013 (and thank you Mr. Musco for endorsing the transparency) .

./tip-hat
+1
2011 early access founder here too. I haven't seen such an in-depth post like this. I too would love for more. I, though not a programmer, have seen and messed with code in modding (breaking games mostly b/c I didn't know code) and understand how cache, servers, client and their interaction works. It's painful to see the complaints by those that don't know how it works, nor how hectic the dev schedule is. This brings so much clarity, and worded in such a wonderful way that's easy for those not familiar with it to understand.
PAX REPUBLICA
Casual PVP&PVE
Star Forge | Captain |International Guild

HKtheindomitable's Avatar


HKtheindomitable
10.22.2020 , 04:03 PM | #199
Quote: Originally Posted by JenHarkness View Post
(awesome post by JenHarkness)
Thank you so much for this post. So awesome! Great explanation.

PLEASE DO MORE OF THESE! "How it works." posts. Too many think it's a push of a button or a few key strokes and you're done. SCORPION and other shows involving "hacking" and/or "programming" are so far off the mark. I would love to see posts about "Why x is limited." or "Why x can't be done." Thinks like hood toggle, chat bubbles, weapon outfit designer, weapon dye. And even like "What can cause x to break and be hard to fix." and "Why does x that seems a simple fix is more complicated to fix than y which seems very complex." or even "Why making x, y, and z is not a quick process." We don't need company secrets. Basic common dev knowledge like what this post has. Like basic knowledge helping folks to understand how the PC model works.
PAX REPUBLICA
Casual PVP&PVE
Star Forge | Captain |International Guild

Alssaran's Avatar


Alssaran
10.22.2020 , 04:57 PM | #200
As a founder in SWTOR and IT engineer, that was actually hilarious to read. Thank you.

Also, appreciate the transparency of explaining exactly what went wrong. It might not be necessary for people to understand that something has been fixed, but it's interesting to read in such a transparent way nonetheless.

If I have helped you, then you could click this link to help me!
A single spark of hope can re-ignite the fires of freedom.