UlaVii Posted October 23, 2020 Share Posted October 23, 2020 Hey all, in case you missed it, there was a great technical post by Jen Harkness the Senior Server Programmer explaining how they investigated the vanishing guild ship bug, what caused the bug and how they solved it. It was very well written and has received a lot of positive comments from the community: http://www.swtor.com/community/showpost.php?p=9889956&postcount=193 Post of the Year imo ^ [quote name=RevivedHut425 ]That's really interesting and kinda gives insight into how unrelated game updates can break different pieces of the game. LOTRO Devs - This is how you communicate technical issues. Please learn. Please can we get more posts with this technical quality? Link to comment Share on other sites More sharing options...
Darkbloom Posted October 23, 2020 Share Posted October 23, 2020 That was absolutely fascinating. Link to comment Share on other sites More sharing options...
DarkTergon Posted October 23, 2020 Share Posted October 23, 2020 very interesting, thank you for posting this Link to comment Share on other sites More sharing options...
FlameYOL Posted October 23, 2020 Share Posted October 23, 2020 I like whenever they come out with the behind the scene stuff of how things works to explain it to us, I feel like if these things were done more often we'd at least be a little less annoyed at bugs since the developers are actually telling us what is going on. I hope to see more posts like these in the future, though not necessarily the bugs themselves, just explanations of how things work:rak_01: Link to comment Share on other sites More sharing options...
DarkTergon Posted October 23, 2020 Share Posted October 23, 2020 I like whenever they come out with the behind the scene stuff of how things works to explain it to us, I feel like if these things were done more often we'd at least be a little less annoyed at bugs since the developers are actually telling us what is going on. I hope to see more posts like these in the future, though not necessarily the bugs themselves, just explanations of how things work:rak_01: I agree, even if I don't understand it all, I can still see the issues at hand, and am probably more patient and understanding of what's gone wrong, and how they are trying to fix it. Link to comment Share on other sites More sharing options...
MadDutchman Posted October 23, 2020 Share Posted October 23, 2020 It's nice to see them posting these details. It helps the non-technical understand and appreciate just how difficult large systems like SWTOR can be, and how one small thing can cause a cascading set of bigger problems Link to comment Share on other sites More sharing options...
orig_mrrabbit Posted October 24, 2020 Share Posted October 24, 2020 (edited) Actually, I learned nothing at all from that . . . ZILCH! 1. What the hell is a "server programmer?" (I can hear my old Dev crew laughing their asses off.) Someone who tells the board BIOS to save it's settings? Someone who installs a firmware patch? Someone who confirms for controllers what it found by pressing "ok"? It's almost like NT Administrators trying to get themselves re-labeled as "Server Administrators" in order to get the same pay and seniority as "Unix Administrators". 2. The flagship bug has been around since flagships were first started. 3. From day one, I've called it what it was - a database error - and I'm NOT an employee of EA / Bioware and have no inside information. Most likely a write error occurred when a key change was made - resulting in the record for the flagship in question getting locked. This is why the temporary fix was to log a toon on the flagship causing a READ of the stored and locked record - pick up a decoration - causing an update of the record read into memory - and keeping it live until the toon actively or passively logged out of the flagship - because it couldn't be saved against a locked record. Also, because it required that the toon in question ALREADY be on the flagship when logged in AND required the toon to STAY on the flagship for the duration - I'm going say it was "STATE" related as it pertains to the flagship record. This above explains why the more permanent fix for years has been exactly as BIOWARE officiailly stated in multiple tickets: "At next server reset." Because resetting the server restarts the database server application initially clearing the locks. EA / Bioware game data centers are PRODUCTION environments. Line employees in production environments typically are NOT allowed to make database changes NOR fix them. An amateur touching a database in a production environment is a recipe for disaster . . . Like any production environment, they have sever "Senior Database Administrators" who are paid extremely well and very good at what they do. However, their time is budgeted on a priority schedule . . . . . . and that is why they never got around to fixing the problem even after 3 years - SWTOR was on the bottom of the "totem pole" for database related tickets from customers. In other words, a Senior Database Administrator was not getting called in unless a ticket indicated something that could be threatening to the game itself. Oh . . . and one more thing. The last defense for a database is a restore from a "known-good" database backup. =8-| p.s. The exact same bug also exists for Guild Strongholds . . . every guild should have a "STUB" toon logged out on their Stronghold AND their Flagship each and every day - permanently - else they're screwed. =8-( Edited October 24, 2020 by orig_mrrabbit Link to comment Share on other sites More sharing options...
UlaVii Posted October 24, 2020 Author Share Posted October 24, 2020 The exact same bug also exists for Guild Strongholds . . . every guild should have a "STUB" toon logged out on their Stronghold AND their Flagship each and every day - permanently - else they're screwed. Heh, I still do that "just in case" Link to comment Share on other sites More sharing options...
MadDutchman Posted October 24, 2020 Share Posted October 24, 2020 Actually, I learned nothing at all from that . . . ZILCH! 1. What the hell is a "server programmer?" (I can hear my old Dev crew laughing their asses off.) Someone who tells the board BIOS to save it's settings? Someone who installs a firmware patch? Someone who confirms for controllers what it found by pressing "ok"? It's almost like NT Administrators trying to get themselves re-labeled as "Server Administrators" in order to get the same pay and seniority as "Unix Administrators". 2. The flagship bug has been around since flagships were first started. 3. From day one, I've called it what it was - a database error - and I'm NOT an employee of EA / Bioware and have no inside information. Most likely a write error occurred when a key change was made - resulting in the record for the flagship in question getting locked. This is why the temporary fix was to log a toon on the flagship causing a READ of the stored and locked record - pick up a decoration - causing an update of the record read into memory - and keeping it live until the toon actively or passively logged out of the flagship - because it couldn't be saved against a locked record. Also, because it required that the toon in question ALREADY be on the flagship when logged in AND required the toon to STAY on the flagship for the duration - I'm going say it was "STATE" related as it pertains to the flagship record. This above explains why the more permanent fix for years has been exactly as BIOWARE officiailly stated in multiple tickets: "At next server reset." Because resetting the server restarts the database server application initially clearing the locks. EA / Bioware game data centers are PRODUCTION environments. Line employees in production environments typically are NOT allowed to make database changes NOR fix them. An amateur touching a database in a production environment is a recipe for disaster . . . Like any production environment, they have sever "Senior Database Administrators" who are paid extremely well and very good at what they do. However, their time is budgeted on a priority schedule . . . . . . and that is why they never got around to fixing the problem even after 3 years - SWTOR was on the bottom of the "totem pole" for database related tickets from customers. In other words, a Senior Database Administrator was not getting called in unless a ticket indicated something that could be threatening to the game itself. Oh . . . and one more thing. The last defense for a database is a restore from a "known-good" database backup. =8-| p.s. The exact same bug also exists for Guild Strongholds . . . every guild should have a "STUB" toon logged out on their Stronghold AND their Flagship each and every day - permanently - else they're screwed. =8-( Um, congrats? Nice of you he discourage them from finally opening up to the community... Oh, and btw, a Server Programmer in this context is exactly what it sounds like, a programmer for the server side of the application. Another way of saying it would be server side back end developer (bit more wordy), as opposed to a front end/UI developer (works with UI designers to make pretty things functional), or a client backend developer (who makes everything not UI related tick (physics, networking and the like)) Link to comment Share on other sites More sharing options...
orig_mrrabbit Posted October 24, 2020 Share Posted October 24, 2020 Um, congrats? Nice of you he discourage them from finally opening up to the community... Oh, and btw, a Server Programmer in this context is exactly what it sounds like, a programmer for the server side of the application. Another way of saying it would be server side back end developer (bit more wordy), as opposed to a front end/UI developer (works with UI designers to make pretty things functional), or a client backend developer (who makes everything not UI related tick (physics, networking and the like)) keyword = programmer Try to rethink my point in bringing that up . . . . . . in all my previous development and integration environments, a person NOT writing code for compilation into executables OR writing code for dynamic preprocessed applications like PHP was not allowed to call themselves a programmer. Junior or Regular Sysadmin maybe, System Integrator maybe, Installation Packager maybe, System Installer maybe, but not a programmer. It's kinda like calling today's janitors Sanitation Engineers. =8-( Link to comment Share on other sites More sharing options...
Recommended Posts