Jump to content

The new matchmaker


HrRav

Recommended Posts

Tonight I have been playing gsf for a couple of hours on Darth Malgus, and matches have been poping very quickly. However, so far the matchmaker seems more or less incapable of identifying skilled players and mixing them into fairly even teams.

 

What is the experience from other servers?

Link to comment
Share on other sites

Tonight I have been playing gsf for a couple of hours on Darth Malgus, and matches have been poping very quickly. However, so far the matchmaker seems more or less incapable of identifying skilled players and mixing them into fairly even teams.

 

What is the experience from other servers?

 

Wait you expected the matchmaker to be able to predict players skills on day one of a new system while it's either using information based on stats players have been accruing while the system wasn't in place?

 

It's going to take months of people playing for their stats to reflect what they need too for it to get it going.

 

The other possibility is they may only be taking into account stats being formed after this patch, which would mean other then requisition everyone started at the same level today and as we play games it's going to learn where to put us.

 

 

I'm super happy with how cross faction is making the queue's pop much faster though and that i can play whatever faction I want now.

 

I've been solo queue ing on Starforge this evening and it was fairly obvious it was struggling to matchmake my new Solo queue alt that only has like 20 games played properly. However other wise it's been a pretty good distribution of players. What was particularly interesting to me was even though I won all my games, my team had more 0 kill players that had very low damage/accuracy then the other team, which means it's definitely trying to off set something.

 

To me it's looking promising so far, but we're going to have to give it time to figure it all out.

Link to comment
Share on other sites

Wait you expected the matchmaker to be able to predict players skills on day one of a new system while it's either using information based on stats players have been accruing while the system wasn't in place?

 

Well, yes, I did indeed expect the system to be better at predicting player skills. They have months or years of data on most skilled players, but yet the system does not seem to be able to differentiate aces from skilled veterans and skilled veterans from average players. From the games I saw yesterday (and I do of course admit that a dozen games or so is a small sample size) it seemed as all these players were given the same ranking and mixed randomly, which frequently resulted in games with one team consisting of 1 ace, 4-5 veterans and the rest average players versus a team with 2-3 veterans and the rest average players. You could already at the line-up see which team would win and the games also mostly ended in three caps or 50-20 victories in deathmatch.

 

I've been solo queue ing on Starforge this evening and it was fairly obvious it was struggling to matchmake my new Solo queue alt that only has like 20 games played properly.

 

This is also a shortcoming of the ranking system the matchmaker is using. We all know that and ace in stock ships will outperform average players and the ranking should obviously be based on legacy and not the stats of an individual toon (or maybe on weighted average of the two, where at least 75% of the ranking is due to legacy).

 

Maybe you are right and the system will improve its performance, but I suspect that it is going to require not just more data but also finetuning of the parameters evaluated by the matchmaker. In particular, I have a feeling that the ranking scale is capped too early and does not acknowledge the vast difference in skill between an ace and an average player.

 

However, I do appreciate that games are poping at a steady frequency. The past weeks (where you have had to wait up to half an hour between games even during prime time on the main European server) have not been enjoyable.

Link to comment
Share on other sites

I had a few fast queue pops, but those turned out to be backfills. Then some very slow queue pops. The most of the wars I was in had a premade of aces on only one team, so the outcome was pre-determined and match-maker cannot do anything as it is to help that. So it will only work when there are no aces, or the aces are not all in 1 group, or there happens to be multiple groups of aces. One war had the 4x top-aces premade, and 2x additional aces, the other side had a few solid players, they put up a little fight but the outcome was never in doubt, maybe some work needs to be done.

 

I do like that I can play the faction I want. I do agree match-maker should examine legacy for stats.

 

If the goal is to try to make the teams anywhere close to even though, unless there happens to be two 4x ace premades, it would have to either split up premade or perhaps do something like a 4 vs 12 (or 8 vs 12 depending). Really, if there is no chance the 4x aces will ever lose, is it really balanced? I'd like to think the aces would like a challenge from time to time w/o having to wait for the rare case when enough other aces queue. It could work maybe with some tweaks like these.

Link to comment
Share on other sites

Surely the only way to solve this is to remove the queue as a group option and only allow solo queuing...

 

Premades are only a problem if there is only one premade group flying on a server and there are not enough solo queuing skilled players around to balance that group (or if the matchmaker incorrectly places the skilled pilots together with the premade).

 

I guess it depends on which server you are at, but on Darth Malgus I have not seen any problems with premades. The unbalance I have seen is simply because the matchmaker places too many skilled players in one team and too few in the other. The sad part is that this occurs in around 90% of the matches....

 

Anyway, although I am almost always solo queuing, I think it would be too drastic to remove the possibility to fly with a group of friends.

Link to comment
Share on other sites

Flying around on Satele Shan, I've seen a lot of premades, honestly. Some of which are so obvious that people have similar names, or names based on an obvious theme (i.e., My Little Pony, ship type, Avengers, etc.). But, they don't pop up often enough to fully discourage me.
Link to comment
Share on other sites

  • Dev Post

Hey there folks -

I would like to clear up a little confusion about the match making changes found in 5.9.2. Here is the patch note for that:

 

Galactic Starfighter matchmaking has been improved to better account for player experience along with their currently selected ship loadout.

 

  • Prior to 5.9.2 the match maker would build a game around the player who had been in the queue the longest. It would then try to add players with a similar loadout score (Loadout score being based around the best ship in a player's loadout for that match). Players were then sorted by faction or at random for same vs same matches.
  • After 5.9.2 the match maker still attempts to build a match around the player who has been in the queue the longest. It still attempts to build matches with players with a similar score, but that score now takes into account the player's GSF experience (Based on their number of matches played across their legacy) as well as their loadout score. The match then tries to balance out the teams to be as evenly distributed as possible.

 

The new system does not have a metric for skill. It does take into account the amount of GSF that player has played as well as how good their loadout is. The matches also now attempt to balance the rosters whereas previously they made no attempt to do so.

 

We will continue to look at the health of the game and the quality of the matches as we go and make changes as needed. Please continue giving feedback!

 

Thanks,

Bret

Link to comment
Share on other sites

The new system does not have a metric for skill. It does take into account the amount of GSF that player has played as well as how good their loadout is. The matches also now attempt to balance the rosters whereas previously they made no attempt to do so.

 

We will continue to look at the health of the game and the quality of the matches as we go and make changes as needed. Please continue giving feedback!

 

Thanks,

Bret

 

The skill differences in GSF are huge and need to be considered to create interesting games. Number of played matches is not a sufficient measure. A simple solution---which at least could be worth a try---would be to multiply number of matches with the kill/death ratio.

Link to comment
Share on other sites

Yeah I agree, K/D ratio or something probably needs to be factored into the matchmaking system because the skill difference is 90% of the GSF balance. Sure I might have played thousands of matches, but Verain or Drako or the like would still wreck me and the rest of my team.
Link to comment
Share on other sites

No, K/D is hot garbage. It's better than literally nothing, but there's entire playstyles, players, and roles it doesn't account for. Chess doesn't rank people based on how many pieces they take per game divided by how many are taken, chess ranks people based on wins and losses. Any system that wants to rank players needs to start with wins and losses, because even though plenty of wins and losses are through no effort of the player in question, some percentage will be. "Wins" is also the only stat you can't cheese by some playstyle that also hurts your group.

 

I don't know how you would try to encapsulate player skill though. GSF obsessively takes notes like "turret assists" (which is perhaps even legacy wide in some hidden compartment!) but has no way to account for positioning, correct use of debuffs, kiting correctly versus kiting incorrectly, or even something as basic as peeling. Heck, the scoreboard doesn't even show hull damage versus shield damage, and I can't even brag about my big-probe healing on the scoreboard because it isn't on the scoreboard.

 

Frankly, in a game that is 8v8 (and now 12v12), even some top sci-fi tier self-learning system would take quite some time to suss out the difference between players near the edges of the distribution. Your bare minimum would be W/L and total games, your better version would be something where you have a matchmaking rating based on the average matchmaking rating of who you are beating or losing to, and above that would be stuff that inspects your actual flying stats, such as K+A, D, etc., as a tertiary note to the aforementioned rating. The GSF matchmaker would have a harder time tracking player skill than a game like League would, and way more than something like WoW's arena.

Link to comment
Share on other sites

Another post, which is maybe in another direction.

 

I'm not really sure what to think about finding out that any part of GSF is legacy-based. I think it's probably better than what we had before, but it does mean that dedicated smurfing will involve secondary free accounts instead of paid subscription accounts. That's a bit unusual given how unlocking alt spaces is a paid token operation from the cartel market (further incentivized by convenience cosmetic unlocks), and as a result of that you now create an alt character who has harsher matchmaking than one you could have put on a free account.

 

In light of this, is it possible to share anything ELSE about GSF among legacy? The legacy-bound fleet tokens were a wonderful start, but could there be some NPC who can be told to remove req from a ship, fleet, or set of ships in some amounts in exchange for these tokens? Or what about cosmetics?

 

This represents a pretty solid shift in the design, from each character being individual to each legacy being individual. If that's the direction, other forms of linkage should be considered to encourage players to put GSF pieces into their legacy.

Link to comment
Share on other sites

This represents a pretty solid shift in the design, from each character being individual to each legacy being individual. If that's the direction, other forms of linkage should be considered to encourage players to put GSF pieces into their legacy.

 

Its not really a shift in design, since they were already tracking those things per legacy anyway. They have to track medals for various GSF-related achievements, which is all legacy based, and they have to track legacy GSF match and match participation for conquest objectives too. All this information was already being tracked, it is just now also being used to make matches.

 

But, of course, I would love a way to take unused requisition from a completed toon and move it to incomplete toons.

Link to comment
Share on other sites

Its not really a shift in design, since they were already tracking those things per legacy anyway. They have to track medals for various GSF-related achievements, which is all legacy based, and they have to track legacy GSF match and match participation for conquest objectives too. All this information was already being tracked, it is just now also being used to make matches.

 

But, of course, I would love a way to take unused requisition from a completed toon and move it to incomplete toons.

 

How is this not a shift in design Phal?

 

Before patch every single GSF character had it's own matchmaking and it's stats. Now your other characters influence each other. That is the definition of a shift in design.

Link to comment
Share on other sites

Kill/Death ratio shouldn't have anything do with matchmaking. Because a player that spends all day getting 20+ assists, by being a healing player, or Ioning or EMP ing wouldn't be tracked at all, this is likely the reason they chose not to use stats like damage done, or kill/death or assists or whatever.

 

However tracking only games played is a really silly way to go about this in my opinion. Especially legacy wide games played. Now if I play on an alt with no gear whatsoever the matchmaking is going to be like oh hey this guy has 12000+ games played I should matchmake him based on that!

 

Even if (and man do I hope this is the case) they weigh how much Requisition you have on your ships way more then total games played it's still a ridiculous way to do that.

 

Either make it character based or make all my ships in my Legacy have all my Requisition.

Link to comment
Share on other sites

Hey there folks -

I would like to clear up a little confusion about the match making changes found in 5.9.2. Here is the patch note for that:

 

 

 

  • Prior to 5.9.2 the match maker would build a game around the player who had been in the queue the longest. It would then try to add players with a similar loadout score (Loadout score being based around the best ship in a player's loadout for that match). Players were then sorted by faction or at random for same vs same matches.
  • After 5.9.2 the match maker still attempts to build a match around the player who has been in the queue the longest. It still attempts to build matches with players with a similar score, but that score now takes into account the player's GSF experience (Based on their number of matches played across their legacy) as well as their loadout score. The match then tries to balance out the teams to be as evenly distributed as possible.

 

The new system does not have a metric for skill. It does take into account the amount of GSF that player has played as well as how good their loadout is. The matches also now attempt to balance the rosters whereas previously they made no attempt to do so.

 

We will continue to look at the health of the game and the quality of the matches as we go and make changes as needed. Please continue giving feedback!

 

Thanks,

Bret

 

Hey Bret,

 

If there any chance we can get someone in the office to explain the WZ pvp match making system as simply as you just explained GSF.

From what most of us pvpers can tell, it is bugged or not working properly, but we’ve had zero feed back about our concerns.

 

Cheers

Trixx

Link to comment
Share on other sites

Presumably Musco's post applies:

http://www.swtor.com/community/showthread.php?t=951165

 

I had assumed that "Matchmaking will more strongly take player skill into account when making teams" would apply to GSF, and we've all been discussing it as such. It's good to know what GSF is doing- I suspect the GSF matchmaker has less to work with compared to the ground matchmaker.

Link to comment
Share on other sites

How is this not a shift in design Phal?

 

Before patch every single GSF character had it's own matchmaking and it's stats. Now your other characters influence each other. That is the definition of a shift in design.

 

I guess I was disagreeing with Verain's use of the descriptor "solid," by which I interpreted it to mean extensive, dramatic, or fundamental … as opposed to another meaning, as in the phrase "solid choice" meaning good or thoughtful. I agree that it is a shift.

Long ago when the pre 5.5 focused feedback threads were active, I think I mentioned something about a ranked GSF system having to lookup legacy stats was prohibitively slow. I was apparently wrong … it does not appear to take an inordinate amount of processing power to look up total games played on the fly to make a match, so it seems like other stats that are tracked on a legacy basis are also fair game, from a computing/database perspective. The existence of the domination game mode, and the existence of healer/support ships, makes kill/death less valid in my opinion, but medals are pretty game-mode neutral and cover a variety of contributions, despite their flaws. Perhaps they could be used in some index?

 

Now if I play on an alt with no gear whatsoever the matchmaking is going to be like oh hey this guy has 12000+ games played I should matchmake him based on that!

 

Even if (and man do I hope this is the case) they weigh how much Requisition you have on your ships way more then total games played it's still a ridiculous way to do that.

 

I don't necessarily agree. It may not be great but its far from ridiculous. There are many in these forums who consistently argue that the skill matters more than the "gear", or the ship and its upgrades, and contend that Ace pilots will win in stock ships just like they do in their mastered ships. They know objective play, they know how ships synergize, they know where DOs spawn, and most importantly they know the importance of communicating with teammates. Total games played in a legacy may not be the best surrogate for experience but its probably more reflective of player skill than the loadout score Bret described. Or are you trying to make the point that you, on a fresh alt with your 12k+ games, are less effective than I with my 4k+ matches over the whole legacy but on my completed main? Maybe I'm underestimating my total games played because with 16 toons I don't have the time to look up the stats on each one, but I can't imagine I have as many as you do. And no one here, myself especially, would claim I am better than you on my main pilot toon then you are on a fresh alt.

Link to comment
Share on other sites

Looking at the new update that is going to happen for GSF it looks like they will have the balance right. If it means faster pop time then Im all for it. Im average player but hope it will put me against people with my skill level when its online.
Link to comment
Share on other sites

No, K/D is hot garbage. It's better than literally nothing, but there's entire playstyles, players, and roles it doesn't account for. Chess doesn't rank people based on how many pieces they take per game divided by how many are taken, chess ranks people based on wins and losses. Any system that wants to rank players needs to start with wins and losses, because even though plenty of wins and losses are through no effort of the player in question, some percentage will be. "Wins" is also the only stat you can't cheese by some playstyle that also hurts your group.

 

 

What we need is something that at least roughly accounts for skill and which is easy to implement in the current matchmaker. Kill/death ratio might not be a good measure as it does not give enough credit to supporting ships and team players (for example those who are focusing on taking and defending sats in domination instead of hunting easy prey). However, I still think that it is unlikely to have balanced games if the matchmaker is focusing on balancing the total amount of played matches on both teams. The win/loss ratio or the mean number of medals per match are probably better alternatives. And it is of course possible to use a weighted average of several of these factors.

Link to comment
Share on other sites

  • Dev Post
This represents a pretty solid shift in the design, from each character being individual to each legacy being individual.

Hey folks! I wanted to quickly respond to this point made by Verain. I do not consider this to be a change in design philosophy. The "experience" factor of the matchmaker is not about the character. It's about the player themselves. To have any reasonable chance of gauging how experienced someone is at the game mode we need to evaluate the entirety of their play experience, at least to the best of our ability. Looking at character experience often give little to no insight into how much Starfighter the player has actually participated in.

 

I hope this helps to clarify this. Thanks!

Link to comment
Share on other sites

EDIT: I posted without Bret's post being visible, so this post was written without his having switched into visible mode or whatever causes posts to travel through time.

----

 

I guess I was disagreeing with Verain's use of the descriptor "solid," by which I interpreted it to mean extensive, dramatic, or fundamental

 

It is fundamental, but neither extensive nor dramatic.

https://www.dictionary.com/browse/solid?s=t

The first 8 definitions refer to physical characterizations, which it obviously isn't, the next is "not flimsy, slight...", the next is "of substantial character". Either of those applies.

 

Historically, an alt has been treated as an entirely new character, with no legacy ties. That was pretty much true, too, up until the legacy ship req tokens appeared, but even that was metered by the relatively slow arrival of fleet commendations, the currency used to purchase them. GSF is filled with alts, and any matchmaker will get confused by said alts without an account tie. Previously, we all sorta believed that such a tie was not really possible technically. There's always been an issue with a legacy tie, and it's not getting the first 2-5 games required to be substantially geared, it is that a dedicated smurf can spawn new free accounts, in the same manner previously relegated to dedicated griefers (who roll free accounts because their main accounts get shut down or penalized). I'm saying that this step represents a shift in design, and sorta implies that other steps, previously out of the question under the old design, are now worth talking about and asking for.

 

However, I still think that it is unlikely to have balanced games if the matchmaker is focusing on balancing the total amount of played matches on both teams.

 

Of course I agree. Matches played is better than "no matchmaking at all". K/D (or really K+A/D) would be better than that. But if there's going to be legacy tracking, it will probably need some kind of hidden "matchmaking rating", that at the very least is updated by going up and down. It doesn't have to be visible to the players (and probably should not be), but an ELO-like design will end up with something a lot closer to a 50/50 W/L for the majority of players over time. The general idea there is, you start with everyone at some rating (which can be the same value, or a guess derived by their current win/loss ratio, or even their current games played, or, at the end of the day, you could literally randomly distribute numbers), and then every game has an expected result based on some "average" MMR of team A, versus the "average" MMR of team B. If A is heavily favored and loses, then everyone on team A loses a decent amount of MMR (with higher rated players losing more than lower rated ones), and everyone on team B gains a decent amount (with lower rated players gaining more than higher rated players). If A is heavily favored and wins, then everyone on team A gains a small amount and everyone on team B loses a small amount. Etc. This is generally how ranked pvp works, except that it exposes the ranking, much like WoW arena and other more competitive type games, and the ranking becomes the thing players communicate to each other. Even WoW arena eventually had to implement a hidden and swingy "matchmaking rating". Since GSF isn't strictly ranked, any MMR in this game would likely be hidden as well.

 

This is a substantial change, likely requiring new development (even assuming the existing matchmaking for ground pvp can be repurposed) and there's no two ways about it, it will never be perfect. Individual contribution to an 8 or a 12 man game varies wildly, and you wouldn't want to brand a visible rating based on what you happen to queue into. But such a thing would pick a good match more often than most setups, and even if you literally initialized with random numbers, you would eventually come up with something resembling reality.

 

Perhaps they could be used in some index?

 

You could, and the advantage of a formula like that is that you don't have to look at it every couple weeks and be like "huh, funny, why is it doing that? What the heck?", like you would with anything that is just driven by data. But coming up with such an algo would be a huge hassle, there's no template for it, and ultimately it won't be driven by wins and losses, and therefore it will probably be less effective in the long term at making balanced matches. Once someone figures out that there's some behavior that allows them to get matchmade lower (and therefore win more), there will some motivation to engage in that behavior. For instance, if kills and deaths are used, a player might choose to focus on assists and crash into a few mountains while still pumping out tons of dps, control, and nest and node oppression. If you focus on wins and losses, you can't really cheese those stats.

 

As an aside, I want to point out that the GSF stats have pretty much always been for fun. Taking them seriously confuses the map and the territory. Matchmaking is kinda serious, and as such I would expect it to not base it on some long formula that needs to be tweaked constantly, but instead based on overarching weighted wins and losses. It won't predict a winner as well as a 1v1 game like chess or anything.

 

And I have no idea if something like that is in the cards. But it could be, given that the ground pvp has something similar.

 

There are many in these forums who consistently argue that the skill matters more than the "gear", or the ship and its upgrades, and contend that Ace pilots will win in stock ships just like they do in their mastered ships.

 

And that's correct. But the argument isn't that gear means nothing at all. In fairness, the current system probably does weigh the gear on your most geared hangar ship pretty decently.

Edited by Verain
Link to comment
Share on other sites

I do believe Matchmaker does need to take into account more then W/L. The W/L % however does not show the skill of the player, but of the team as a whole that player is in. For example: Player A solo queu's, he is more dependent on the random 7 or 11 other players and may have to carry more weight to even have a chance at winng. Player B like's to fly with friend (which is 100% fine), but because of this he can communicate easier (if using chat), and coordinate with his team mates so winning a match will generally be easier.

 

With that being said, all of the stats need to be looked at to see player skill. BW has had in place character tracking of stats for a long time, and can see generally how well most of us can play. K/D should be part of the formula, but not all of it. They should also look at Average kills, average assists, Powerup gained (this is a good one for skill, the better pilots always snag these up), Sats captured, turrets destroyed, Time Defending sats (Should be looked at, don't think that's calculated), and Medals earned. Clearly most of the would be character based, so each character would probably have a slightly different rating, but it's the only way I can think of that would be far better at grading everyone.

 

A character with 0 matches on it, should have that in consideration (but still count legacy against it for matches) when grading it. The pilot may be decent skill wise, but lower graded ships to have disadvantages which most pilots don't completely compensate for. You can be very effective on them, however generally far less then usual.

Edited by Toraak
Link to comment
Share on other sites

Hey folks! I wanted to quickly respond to this point made by Verain. I do not consider this to be a change in design philosophy. The "experience" factor of the matchmaker is not about the character. It's about the player themselves. To have any reasonable chance of gauging how experienced someone is at the game mode we need to evaluate the entirety of their play experience, at least to the best of our ability. Looking at character experience often give little to no insight into how much Starfighter the player has actually participated in.

 

I hope this helps to clarify this. Thanks!

 

My previous post was written without access to your post, which popped into existence from some wormhole while I was moving my trash cans around outside.

 

Previously there was no legacy look-up, now there is, right? I view that as a design change, even if your intent hasn't changed. I certainly don't dispute the rest of your post. Regarding the body of the post that offers justification about the change, I'll agree and go further- I suspect that from the dev perspective, this game has been descended upon by a plague of alts. Certainly from my perspective, it has been. I have like 20+ alts that pop in and out of GSF at times, probably, across the three servers that pop GSF. I constantly see characters that I've never seen before playing like players I have seen before, etc.

 

I feel pretty motivated to roll GSF alts, and others apparently do as well, as the barrier for gear is extremely low, and the pleasant lack of harassment (and separation of stats, for those who focus on that) are pretty substantial rewards. The costs are a few games spent undergeared (or zero games if you are willing to spend all your fleet comms), and the need to hide your character somewhere that a nosy person can't inspect them (legacy/achievement inspection is a serious invasion of privacy, imo, but that's a whining rant for another day).

 

The side effect of this swarm of alts is that any matchmaking is going to get totally ruined unless it looks at legacy. Once it looks at legacy, it will have a much better way of making matches, because the only people that won't work on are dedicated smurfs.

 

My point is basically, if legacy is going to be used for more stuff, push for more legacy links, such as requisition and cosmetics.

Link to comment
Share on other sites

×
×
  • Create New...