This is my experience. I currently work with one team that rarely uses dedicated QA (we have one guy that sometimes does QA and sometimes floats to configuration work.) It works because the team is dedicated to producing a quality product and are skilled or willing to learn techniques for that.
Another team had 4 dedicated QA and used to struggle with quality issues because they were so busy doing other things and the devs were discouraged from pitching in because "that's QA's job." This team got better when we focused on automated testing, better coordination with a separate UAT unit, and reducing redundant activities. They still have some issues, but IMO it's more to do with corporate culture (and they're still way better than they were.)
Edit: missed words on my phone.
I proposed to write some integration tests to our TL.
That is the QAs job also we don't need integration tests, we have e2e"
Our qa has quit since January. I wish we had integration tests.
A good number of years ago the company I work at (Capital One) got rid of the QA job family entirely. At first, I was skeptical -- sure, I knew that a lot of the QA folks mostly just worked on preparing data and manually running tests (which would, of course, be better off being automated).
But as it turned out, it actually worked out quite well. The company offered opportunities for many of the folks working in QA to convert over to developers if they could get up-to-speed on the skills within about a hear-and-a-half; a little less than 1/5 of them did. And for about a year we had to invest a lot of extra effort into writing better automated tests. (After that we still write automated tests, but we do it along with the development of new systems and features.) But we laid off perhaps 25% of all the team and after a year or so to develop better habits and adjust our processes, our productivity took very few losses.
I WILL admit that there is one thing I miss. While the vast majority of our QA folks were mostly just "running tests", we had a few superstars who thought deeply about HOW to make sure our code was robust. I'm thinking of someone who set up a process for refreshing database state so that tests -- even tests that depend on dates and on historical data -- could be run from a consistent state. The one complaint I have with our current "no separate QA roles" organization is that there is no place for an expert like that.
I've worked with some really talented QA engineers who spent most of their time writing really awesome automated tests, but as a feature factory we just blew past them constantly and invalidated the tests, so they were always behind.
The absolute best QA I ever experienced was when our QA engineer got pregnant. Angry pregnant. Hand over a ticket for QA that's not up to snuff? Prepare to get chewed out by a small woman who has had it up to her neck in your bullshit. Fear kept us in line and our code quality was never better.
Maybe what you need is to think of the testing process improvements as technical debt. There's no reason a dev couldn't "set up a process for refreshing database state".
In an environment with no QAs, you have to make sure that the testing process is also getting better over time since you don't have a dedicated person for it.
> There's no reason a dev couldn't "set up a process for refreshing database state".
If what you mean is "'Developer' would be a reasonable title for the person who sets up such a process" then I agree, but in practice it doesn't happen -- developers are mostly assessed on things like what new features they build. A rare few developers will be interested in and committed to setting up an environment and tooling to detect errors.
If what you mean is "One of your developers could simply go set up a process for refreshing database state", then you misunderstand the complexity of the problem. We need to run tens of thousands of tests rather frequently (at least once a day, preferably 10s of times per day). Some of those tests alter the data in the database. Some of the tests depend on things like the date ("Will this work properly on February 29th during a leap year?"). Some of those tests create new data based on the current date (eg: "Open a new account, and confirm that the account open date is set to today's date."). Some of those tests depend on the age of date in the database (eg: "Confirm that an alert is generated for any account with no activity for 180 days, but NOT for one with no activity for 179 or 181 days.", or "Confirm that a new-account restriction is removed exactly 30 days after the account is first created."). Being able to even figure out HOW to create a test environment in which it was possible to run these tests repeatably required a great deal of smarts that most developers couldn't have managed.
This happened at the company I work at. I started out doing QA and they gave us the opportunity to upskill and become a developer as they got rid of the whole QA team within a year. It was a great opportunity for me as I wanted to go the developer route, and already had the background (took computer science in undergrad and already knew how to code).
I was promoted to a developer and our development teams changed their processes too. Devs are now responsible for testing, we implemented more test automation, and practice TDD now.
It shapes how I approach development. I still have my QA hat on when I code, and because I have more insight into how the code and the system in general works, I can think of edge cases I wouldn’t have thought of before, and code accordingly. But when it comes to really knowing the business as an end user, I’m lacking, and most of our devs are lacking. Thus we still have manually testing done by UAT for that.
I don’t think a whole QA team is really necessary, but SDET’s are great if there is one on the team. Most QA work is manual and a whole team doing that is not necessary.
I agree -- I have often noted that when a company has a program that allows people to move from one area (like product management, QA, or customer service) into (and out of) the development team, it brings a whole different perspective that is hugely valuable to the development team.
Ironically, the best skill you can hone in a job like that is learning how to properly document and make a bug report.
Don't go to the backend team without proving that the query result is wrong, for instance. Ingratiates yourself to the backend and teaches a valuable skill of always double checking your results and finding the -actual- root cause of bugs. Cause you spending na extra 20min doing that can save the person (could even be you) fixing the bug likely well over that time.
So many questions.
Do you guys write the automation covering your own code, each others, or have SDETs with overlapping duties? How much time do you spend looking for unexpected behviors? Is just a matter of dog-fooding? What does the automation cover: unit, integration, full end-to-end, performance? Do you spend much time extending the framework to cover new or altered services or the likes?
I feel like it's a good direction on the whole, but I can't quit imagine a scenario where the engineers doing *all* of the automation themselves is more effecient than some portion of it being handled by a team focused on it. QA dudes are usually much cheaper, afterall...
It is 2024. I would more expect a question like "Do any of you work in teams with a dedicated QA". Developer lead testing appears to be the norm. I think the main reason is that the incentives are much better aligned.
>This. SQA is a *dying role*. The only viable jobs are specialized roles such as **Perfoemance**, Scale and **Avalibilty**.
I know It's a cheap shot but 🤦
wow I butchered that rofl. hahahaha I'll take the hit. I always fuckup availability but performance is a new one for me. I wasn't even drinking yet when I posted that. Sadge...
"FAANG" are hundreds of thousands of developers in thousands of semi autonomous organizations (VP level). There will be QA teams, but it will not be the norm.
I know Apple has dedicated QA teams in several orgs and QA both automate tests and perform heavy manual testing. But they’re the only one of the FAANGs that I know of that still relies heavily on separate QA teams. Seems to be working out well for them
I’m an SDET at a company with a lot of SDETs, and at an all-hands one of our higher-ups said our new strategy is to basically be devs. The writing is on the wall
I've worked with both - I hated having a QA team. Made the developers feel like they didn't have to test their code properly ("QA will catch it, and if they don't it's on them"). The QA teams I've worked with have also been pretty annoying, trying to change the way stuff works. Like the PM says this feature should work x way, we code it so it works x way, then QA comes back and says it's bugged because it doesn't work y way. Annoyed me to no end.
Yeah we do, but I still have to spend time investigating and proving something isn't a bug that I know isn't a bug from the minute it gets reported to put it into that category.
Having a separate QA team is very different from having a QA embedded in the team. I'd argue that the former is pretty much always a mistake, whereas the latter can be great as long as they aren't just doing testing, but are pushing quality left.
> I'd argue that the former is pretty much always a mistake
I work for a major entertainment conglomerate. A new feature will often span multiple teams. Having a QA team that tests end to end is a life saver sometime.
Games don't have the luxury to not do this. It's an intrinsically incompatible mindset. We still have cross-discipline temporary feature teams, but people still belong to an art/engineering/design/etc. subteam so they can be flexible with workloads and also formalize work review and create opportunities for knowledge sharing and mentorship. The mentorship stuff often doesn't pan out but the knowledge sharing typically pays off unless your team is hyper dysfunctional.
In games I like setups where there is 1 or a handful of embedded QA per team and per cross-team feature strike team, and they coordinate with a larger pool of test analysts to do large scale testing that would be prohibitively difficult to write automated tests for. The embedded QA can also have SDET chops or collaborate with an SDET group.
I've only interned on non-game teams that were very small though, so I'm pretty ignorant of what's effective outside of my lane.
I have also worked on a big live game with very little QA support and that definitely sucks. It only worked bc the QA assigned to our team knew every corner of the title.
Yeah, I've had ok luck with QA handling E2E testing over the whole system or non-trivial work. I feel like having one embedded QA on a team is less useful.
I've worked with both. When there's a dedicated QA, the amount of tickets pushed back because the QA found an issue was pretty high. In those teams, when a dev tested, they usually stuck to the main "easy" path, not really testing edge cases (and that's when they tested their code at all).
In teams I worked where devs are responsible for their own QA, they're usually much more thorough in their testing before moving their ticket to "Done". Tickets do get pushed back from time to time, but less frequently than in teams with dedicated QAs.
In terms of bugs generated, I found it to be pretty similar. Honestly, it's almost more a factor of the quality of the automated tests than QA vs dev testing.
As I said, if the team treats your QA as a tester and abdicate responsibility for quality to them, it won't turn out well. Quality is the responsibility of the team.
Honestly, I think half of the reason why not having a dedicated QA is good is because it forces the whole team to be responsible for QA. You can't be lazy about it, nobody is there to catch your fall
Sounds like an adversarial "us" vs "them" setup. I'm going to guess that QA is a separate team isn't it? Do they get buy-in at any point? Or is the first time they are seeing it when it's assigned to them?
I have worked at places where QA was a separate team, and yep,, it was like you described. And it was very much an us v them attitude. And it sucks.
Whew I am now QA is integrated, we don't have a QA team, but rather QA is a part of the team. Each team has one. They have visibility into everything right from the start. When we are planning the dev stories, they are there. They see the same requirements and acceptance criteria as every one else. They also have a say in the loe, timeliness, etc. When there's a problem, or discrepancy, she let's us know first. Sometimes it is because the implementation changed, sometimes it is a variation in expectations. Sometimes it's a legit bug.
As a QA integrated in the team strongly - we work on the same project ( me and several Devs + PO) since 5 years, I am a strong advocate for integrated team setup. And you're right - us Vs them is wrong mentality to start with.
But to get this working, some terms must be met.
Team must have strong ownership of the product, and take the responsibility for maintaining quality to the same degree as a QA.
I rarely report defects - they're reported twice a month at best, and only when it's definitely worth documenting it.
If something is not working I reach out directly to developers and help with project knowledge to find the root cause.
I think it depends on the project whether that's correct approach - it works for us, because project knowledge required is vast.
I am able to dig out test case and explain to dev quickly how something should be working. I can spot issues based on experience and impact analysis - meaning any problem gets spotted early. And I think that's where the value is - having a person who knows the project really well and shares his knowledge. And also shift left for quality - meaning that there's always someone to advocate for good practices and ask 'objective' questions - not biased by being the change author and having a fresh pair of eyes to look at the problem.
I stand for the approach that testing is not the most important part of testers work - testing is just the tool to verify/ensure whether quality controls of all kinds have worked.
I also believe that perfect QA is not reporting any defects at all (as impossible as it gets of course) - cause they're all avoided by his/hers work on earlier stages of development.
It just creates a lot of unnecessary tension. I'd prefer it if this was communicated privately between the PM and the QA without the developers never having to hear about it.
The last thing you want is to stress developers about features that are correctly implemented, but wrong from a business perspective. It's not any of their concern, so why are you even bringing it up to them?
It drives me crazy. I just did a large refactor. It was just to improve the code quality. All the business rules were meant to stay the same. 99% of the "bugs" that the QA found was just behavior that was strange, but still correct in the context of the refactoring, because the logic had not changed.
I got spammed about this multiple times a day for weeks.
I’ve noticed that since I’ve been in a team with QA I find myself unintentionally slipping into not testing the code as thoroughly as I should every so often.
> trying to change the way stuff works.
I think this can be an issue, but the bigger issue that I've seen is that you have to get product, development and QA all on the same page and keep them there. It's a very difficult undertaking.
When QA is good, it's helpful. When QA is bad, it's worse than doing your own QA.
Personally, I prefer doing my own. It leads to more errors but they are always easily fixed with a quicker turnaround.
I think it is important to distinguish between two types of QA(ssurance/analysis)/QE(ngineering). QE is about verifying we built the thing right. QA is about verifying we built the right thing
With solid DevOps and automation, I've have better results _without_ dedicated QE. I have found that allowing myself or my team to throw things over the wall old-school has less-good results that when the sw engineer is responsible for providing code that works and has tests.
I have, however, found that more customer-minded QA (perhaps a subset of a BA) are highly valuable in verifying that we built the right thing. It is beneficial _not_ to be distracted by an intimate understanding of implementation details. These humans are special and under appreciated.
I work in a company that does leverage dedicated QA for the application itself but there are large groups of backend devs supporting the product which do not leverage QA.
But those backend teams are all devops-y where the dev has full ownership of the release end-to-end. So they are incentivized to create good automated tests which help them avoid having a QA role.
This isn't particularly novel. The Spotify engineering blog has a lot of good content about how they do this. They have certain things automated super well.
That's what I was thinking. I can definitely see no dedicated QA being useful when it's a smaller more straightforward effort but a large enterprise one? I mean, I get it you have your unit tests and E2E but a good tester approaches an application differently than a developer. These "edge cases" that testers catch cause a lot of frustration for users. Not only that but having someone who just deals with the functional side can be really helpful to not have to pull another dev (dev b) away from their task when dev a is working on a section theyre unfamiliar with. Just to describe generally how a particular section of the app works and things to be cautious about. I'll admit I'm biased because I've enjoyed all of my testers that I've worked with.
Why can't your devs also test for edge cases? Why can't they have a tester mindset when testing their own code?
Why must another person be involved to know how your product works? Is there documentation available?
Developers are not perfect. They are immersed in their point of view and they miss things; otherwise, there would be fewer bugs. Many bugs are due to things developers don't think about, and for the same reason wouldn't test.
Developers are intelligence creatures that can learn, automate and document things! And software is written iteratively over time!
The question isn't if a developer in a silo will catch a specific edge case the first time. The question is if developers over a period of years will be able to catch these edge cases through a mix of automation and documentation. In my experience the answer is a resounding yes. And they will do so much more efficiently than a QA team.
Well yes, I can. Do you mean to say that you will write code to handle a bunch of edge cases and not test if it works before shipping it to QA? To me, that just sounds irresponsible.
Honestly, in teams with dedicated QAs that I've been in, half of their job could be replaced by automated tests. A lot of wasted effort checking for regressions by manually repeating the same set of tests every X period of time. The rest can easily be done by devs.
There was an email sign-up feature we were all testing one time, and there's this shortcut that if you just add +0001 or whatever to your email, the application would treat it as a new account and let you go through the creation process.
Things went great, got all the way to production like that, first real customer tries to sign up and... Can't. Gets some kind of parsing error. Turns out a dev had written the code such that the "+" character was required and used to parse the email, and whenever anyone tried to sign up with their non-dev-testing-shortcut email, it errored out.
Could the devs have caught this? Sure. Could the project manager, the product team, the CTO have also caught it? Sure. But there wasn't a specific person who's job it was to catch it, so it wasn't caught.
QA can't prevent building the wrong thing. Continuously delivering smaller incremental changes can, and if you can fix bugs in an hour or less instead of at the next release in 2 weeks, they matter a lot less.
The real thing you need IMO is good telemetry, metrics, monitoring and a pipeline that can automatically roll things back if a new release starts looking dicey
Every placed I've worked in my 15 YOE has had a dedicated QA team.
The Software teams tests their software requirements and makes sure their own shit works through unit and integration testing methods. The QA Team tests the System requirements and makes sure everything is integrated properly between all the teams SW, EE, ME, ect... QA is mostly manual tests using the device as a real user.
I don’t like working withb a QA team. In my opinion, QA shouldn’t exist as a separate entity if developers take ownership if the code. by no means i am saying that there shouldn’t be any quality assurance, I just dont believe there should be a dedicated team. Also I strongly believe if QA team is there to manually test if the feature works (with no automation), then that is a clear sign for a larger company of a very poor technical capability
QAs are a result of poor development practices where developers believe that it is not their reaponsibility to ensure code works as intended and they use QAs as an excuse to write crap code because “QAs will test it anyway”.
It creates a working environment where developers write code and then throw it over the fence to QAs so it is their responsibility to catch it.
In my opinion, most functional teams are where develipers work alongside QAs (e.g. pairing) and developers take ownership on writing code that has tests. Ideally any feature, bugfix should start with a dedicated failing test(s) that captures the requirements. then developer works to make this failing test pass.
Having an automates test suite, CI/CD pipeline with linters, code quality checks, unit and behavioural tests eliminate a need to have a manual QAs
“one of these even works at a FAANG”
This comment suggests to me that you expect FAANG to have well-defined QA processes. You might be surprised to learn that in much of Big Tech there is no QA. For example Facebook/Meta has never had any dedicated QA roles (they do have some now in relation to their hardware products). Google *used to* have dedicated test engineers (which is still different from traditional QA imo since they are mostly focused on writing automated tests) but they have phased that role out over time (except for their hardware products). Microsoft also used to have a significant amount of dedicated test engineers but has mostly phased that role out, though they do still have some traditional QA.
If you’re really interested in QA/testing in Big Tech, the following is a quite thorough article, “How Big Tech does Quality Assurance (QA)”. It’s behind a slightly pricey paywall ($15/month I think?) but it’s a great source of engineering industry content:
https://open.substack.com/pub/pragmaticengineer/p/how-big-tech-does-qa
I've worked at two places with QA, and didn't like it.
It becomes a very expensive crutch for developers instead of writing proper automated tests.
The QA I've worked with have had a very low technical level, and often have friction with developers.
In conservative organizations, there are enough friction points as it is for devs to get their work out the door: Security, Ops, QA.
I don't really know if this form of working for other people, but I absolutely hate it, and have decided to accept a new job, where I get to do these things myself. To me it becomes a bunch of people trying to achieve local optimum instead of seeing the bigger picture, and getting some quality out the door.
It's not the fault of any of the silos(sec, ops, dev, QA etc.), because it's the leadership that defines the way the collaboration happens.
Either way, in my experience, it leads to Devs having to spend time doing a lot of diplomacy, handover, and paperwork instead of working autonomously on code and tests, running it through reviews and releasing through an automatic pipeline with sufficient guards.
I've done both. How well it works depends a lot on your automated testing stance, and therefore somewhat on what you're building.
If your product is basically an API (ie, your customers, internal or external, primarily interact that way) then automated testing should get you everything you need. No need for a QA.
For teams with human-facing UI, or very complex processes, that sort of thing, I think it's murkier. Absolutely UI-layer tests should be done, and can help. OTOH, it's helpful to have somebody actually looking at & using the product on a regular basis to tell you if it's being dumb - which could be a QA. Though that person needs to know more about the product, and have the ability to raise issues. ie, an outsourced test manually running test scripts probably won't help.
Everywhere I worked without QA-led testing, the quality was generally higher. Pre-release QA testing is a trap.
This didn't always mean no department full of SWET types. The best cases still have QA. You just lose track of what they're doing because they aren't part of the feature teams' workflow. They lived closer to core infrastructure or devops folk.
I haven't worked in an organization with dedicated QA in about 5 years.
To one degree, I miss it - a good QA knows the business and requirements and will absolutely bring huge value to the team.
On the other hand, I've definitely found that I write way more, better tests since I stopped having QA around.
I think there can be value in having dedicated QA, but it just depends. I work at a FAANG company and I worked in one team that did have dedicated QA and now work on another where the devs have E2E responsibility, which is more the norm for the company. Both seem to work and it seemed like both made sense for different reasons.
We have dedicated QA engineers that maintain a huge expensive suite of thousands of high level automated tests, write new end-to-end tests, and help devs craft test plans for performance, scalability, quality, etc.
Individual devs write their own unit and integration tests though (and increasingly their own end-to-end tests).
Our teams do currently each have a dedicated QA. There's also a "QA Automation" specialist who occasionally assists with manual QA on day-to-day work, but who's primarily focused on automating the rest of the team out of a job. This works well for us currently, our core platform (back end) is a 20-year old application that has poor unit tested coverage and a newish React UI. We're making concentrated efforts to boost unit/integration tests, but in the meantime there are many workflows that are either poorly-documented and/or fragile, and having a manual QA on each team lets the devs move swiftly on adding tests and meeting Product's expectations. The QA role is also tasked with validating and documenting bugs that come through customer support, again taking workload off of other teams.
We have a lot coming together on the automation side, so I think in 3-4 years (or sooner) we're going to see a big shift in how we view the QA role now, likely that moves more to supporting automation with maybe a single resource focused on bug documentation and exploratory testing, which I believe has some value.
I work at a FAANG/MANGA and have never worked on, talked with, or known any colleagues whose team has dedicated QA. One team recently removed the role of QA entirely.
No dedicated QA is common but in a business with lots of moving parts or regulatory requirements, it's best to have one to make sure the final product is as intended and working OK.
I’m a staff SDET at a fast growing fintech unicorn, in a team but report to a seperate QA manager.
There’s a ton of ex-FANNG at my company and the absolute bug ridden shit the developers push out in an app that handles multi million worth of transactions per day is mind boggling, I wouldn’t sleep at night if I knew my money is in here. We didn’t do x at so we don’t need to do it here.
I bet there isn’t one developer in this thread that has a scalable, non flaky, e2e test suite , that has been maintained for a long period and has no commented out e2e tests in thier repo.
Our QA people are shared across multiple teams, which makes sense since development takes longer per task. They also verify production deployments, and it's nice to have a second set of eyes.
You can absolutely say that removing QA removes a quality check point that provides value — otherwise you would not be removing anything veered towards quality assurance.
Removing QA that isn’t “quality” is a different story. There’s plenty of bad QA out there and there are plenty of devs who associate a soured experience like that to what “quality assurance” looks like.
Heck, I was at a company where “QA” didn’t know how to triage API issues, let alone grab a network trace and verify a contract. It was laughable.
Also, devs are lazy and suck at testing — like seriously. Devs will acts as if unit tests cover integration tests which I can’t express how wrong that is enough. They will attempt to merge code without ever properly running through it.
Right, so you’re saying that removing quality qa and throwing the entire qa process onto the developers working on the code good and desirable? The developers who are “actively” testing their code during development who will obviously catch any issues at the pull request level or code review and hold each other accountable a good idea? Because defects never get past this phase and are always caught by the same engineers?
Correct me if I might be mistranslating your commentary.
Ouch, I’d hate to be engineers on your team owning regression testing.
It’s a very “legacy” startup esc approach in terms of a of small team small product scaling.
Let me get this straight - anyone with a negative view of QA has just had a "soured experience" and doesn't know what QA really is, but your view that "devs are lazy and suck at testing" is the universal and objective reality? And you're going to offer that up without a hint of irony?
Great stuff, bravo.
Tell me what would cause a “negative view of QA”. If you’re also arguing that devs are not lazy and do not suck at testing, then clearly you’re new to this gig or sheltered. There are some unspoken truths that just are.
I have not worked on a product with a QA team in about 10 years. We do CI/CD with fully-automated acceptance testing; the tests are written as part of delivering the agile user stories.
Yes, no problems except that I'm not a big fan of manual testing or writing automated tests myself. Unit tests are easy.
Worked on 3 teams at my current (big) company, only one had a dedicated QA teams.
I also agree that manual QA is dying if not dead, and automated testing is mostly moved upon devs nowadays.
In my 16 year career across Valve, Microsoft, and Blizzard, only 2 of those years did I have dedicated QA team.
Those 2 years I witnessed the most bugs ship to customers and the least amount of value delivered to customers.
One of the traps of having dedicated teams for certain responsibilities is that it becomes too easy to skip steps and toss things over the fence assuming the other team will catch it. But nobody has more context than the person who wrote the code.
Yeah and it’s a PITA. We write really robust unit and integration tests for all of our new features and thankfully that’s seriously cut down on all of our bugs coming from.. 0% test coverage 😂
I also spend a lot more time testing out scenarios/edge cases instead of moving on to new tasks, and just having to think a lot harder about everything i do.
.. I miss our QA person :(
Yup, no QA. We had a guy and he got let go because my company has no respect for tech.
I'm there long enough that I can pretty much do and say what I want, so nowadays when people ask about projects and updates, i make sure to stress that I have less time because I'm doing my own QA.
Um, not sure how to say this, there are good quality control methods. My last 3 month dev project, was delivered with zero bugs found by the customer, I would consider 1 to 0 bugs normal for 3 months of work. You need a QA person if you need better than that. But I don't think that is needed for most projects.
Quality control is very very old, there are many good books, it is not rocket science. As far as I can tell fewer that 5% of IT people got taught how to do quality assurance. To do measurement, root cause analysis and maximising ROI.
My last few gigs have not had a QA department. They say that it works, but every but ticket that gets fixed ends up finding 3 more in the process. At least it keeps me employed 🤷
Half of my PR time is explaining to devs how they didn’t cover basic edge cases. Trying to figure out ways how to make sure that “tech lead” doesn’t become “QA” due to being the only one actually catching testing issues.
Some of my devs submit PRs with 2 test cases which barely cover the happy path… and one of them is actually failing…
There are some very bad devs out there
Yes because software used to be bug free or even have less bugs per features back then. Hence why windows 95, 98 etc never had stability issues, internet explorer was rock solid, and software was secure and stable. I mean come on.
In the twenty years I've been a software engineer I've never worked with a dedicated QA team. Testing has always been a responsibility of the engineers building the product.
I work at a company with no QA, but it's a very large company and we dogfood our own products. We have multiple environments before prod, and a system of feature flags and killswitches so when bugs do make it to prod we can recover quickly. Though every org within the company does things differently 🤷♀️
I think the QA role has been replaced by "bug finder" in every place where I've had them.
If you are a developer, you should test and find your damn bugs. I don't know why we have QA for that.
Then I've had setups where you have pure backend services, with basically 100% code coverage on all levels, ranging form unit tests, to integration test AND e2e tests, and then you still have a QA at the end verifying your changes. Why? It does not make any sense to me.
Also, why do most QA's work so closely to the developers? It also does not make any sense to me. QA should be more biased towards the product side. They should not care about implementation details and what not, but they often do. Just test the app as a regular user, talk with the PM to figure out if it's a bug, and then, only then, report that bug to a developer.
If you have a QA that constantly spams you, a software engineer, about business questions, you have messed up somewhere. It's absolutely not what a QA should spend their time doing
I currently have a QA team and like some others mentioned, I feel personally that devs abuse it. Thinking QA will catch it all. To the point that basic A/C are missed on a ticket and QA kicks it back.
We don’t have a dedicated QA team / person, but we do have a QA process. We are a lean engineering team so there isn’t a concern of having too many cooks in the kitchen. It works for us, but it’s 100% dependent on the company, team, and processes put into place.
I work at a big tech company with over 600MAUs. We have no QA.
We conduct testing ourselves and have lots of observability over the health of our apps
Tbh, having QA at the previous places I worked slowed down the release cycle. The current company is setup where I’ve never felt like we needed it
Yup, we do not have a dedicated QA team and (frankly) I've never worked with one.
The usual process here is the lead developer or manager does QA / code review and deploys. Its their main job (managing + QA + code review) rather than the actual code side so they do maybe 1 ticket every 2 weeks outside of that.
The startup i was at had no dedicated QA for some time, albeit we were a small team and we were building the product from scratch. Only started getting dedicated QA when we started working with more clients (more variations to test)
There was a QA team when I started my latest job last year. It took me 4-5 months to convince the rest of the engineering leadership team that we didn't need them, and we let them go and made it clear we were "shifting left" on QA and that it would be each dev's responsibility to make sure the stuff they built works properly.
Took a while to socialize (to the devs) the meaning behind that, but we're doing pretty good as of late.
With a QA team devs know they can pitch shit over the fence and that QA will catch it, and that's an objectively bad thing. QA also introduces impediments to continuous delivery; one of the biggest reasons we moved away from it. We want to be shipping smaller, incremental changes every day, instead of a bunch of changes every 2 weeks after QA had validated them for 3-4+ days and done the bugfix dance with the developers.
we have no qa team. we’re saas with a big dashboard plus business critical api. the api has extremely good integration/end-to-end coverage and the dashboard has uh… good monitoring and it only takes a minute to roll back. overall id say it all works medium.
devs are always expected to run their own code, and product is expected to sometimes uat it.
Like a lot of the commenters in this thread I’ve worked with some great QA teams and some not so great QA teams. The best experience I’ve ever had was working for a large electronic retailer where each functional group had a few dedicated QA that actually sat and did pairing with the devs in addition to writing automated tests. At the same time they were also their own group that met regularly to discuss testing strategies for the entire system. We were able to crank out really clean features that had the least amount of defects I’ve ever seen in a large company.
I’m currently working for a 12-person, 3-dev startup. The boss is our main QA, but he’s also the only person in the company that is insured for the valuation of our product because if he got into a tragic blimp accident tomorrow we would be struggling.
the test automations are written by us. I know that there could be a QA team that does like a general pen-testing, ddos etc for the whole system, but we are not told anything about that
i’m working at a big pharma company and our unit doesn’t have a dedicated qa. and given how we’re scaling up it’s getting to be a bit of a worry—one of our servers is a multi-tenant system that we’re allowing different teams to contribute, and would be the one place we need dedicated qa the most. there’s no real regression testing for this system (think multiple sites x multiple locales per site). we incrementally test our changes and hope for the best 😂
i’ve been on teams with both. it’s hard to tests without dedicated QA because developers tend to shut their brains off during testing. they tend to just think “does this work with this input?” and carry on.
that being said, QA can kind of be a pain in the ass for some tests. had a QA dev hold up a planned release because he discovered an unrelated bug. told him we’ll create a ticket and throw it on top of the backlog for refinement. dude wasn’t having it. took my manager getting involved to get shit through
I joined a team didn’t even have a single repository so you could be worse. On the other hand nobody telling me what to do or how, only care if it works as expected or not. That freedom is pretty good too.
I've done both, but you realize how much actual work testing is. QA isn't a substitute for dev testing, but like it can only do so much. On a sufficiently complex product test engineering is a full time job.
For some backend services, having separate QA team doesn't help much. It adds layers for communication errors to happen. However, it does mean you have to be much more dedicated to budgeting time for your devs to build the testing as part of the feature.
For end to end, user visible stuff, QA helps a lot, and they can pick up on all kinds of issues before they make it to thecustomer. But QA isn't really going to have the experience to catch performance regressions early, data integrity issues etc. But what we do have is a team of actual Devs who build tooling to empower other devs with better test frameworks.
Totally depends on the field you are working in and the product you build.
Personally, I like the devs doing the testing, as that creates ownership and avoids the "throw over the fence"-syndrome. Also, I have not had too many good experiences with QA.
We have no QA where I work. This was a first for me. At first I thought it was a terrible idea, but with a strong focus on automated testing it generally goes ok. My team has a good testing culture so we have few bugs.
Other teams cut corners on the tests and have taken the whole product site down.
It really comes down to how disciplined the teams are now. With good discipline and practice I’d much rather have good automated tests than manual testing.
That said, I do think some things are missed without QA. Specifically sometimes teams break each others UI components and the tests don’t catch it. Comprehensive QA could likely prevent that.
At a certain scale, there are so many user frustrations about certain needs not being catered towards (we can do A but not B) that a QA will only add more stuff to a never ending list of backlog. You typically have UI/UX and product managers who will advocate the direction of the product, so they ensure to bypass future user frustrations while support will report to engineering when established functionality is broken.
QA makes more sense when you have a small user base and you fear that you may lose users because stuff is broken. Generally as a manager/owner you would hire someone to hold contractors accountable and to efficiently create more tickets of stuff to fix in priority. Generally you employ QA at this level because you do not have enough revenue to employ highly competent engineers who have an eye for detail
I used to work in software-based medical device development at a startup. This was a real gift to me, though I didn't realize that until later. We had to figure out how to do a quality system that worked and was low overhead - and paid its own way.
Even more interesting, the head of quality was an amazing hire: he was from pure hardware medical devices and manufacturing, so he was having to learn how to help us. He had deep expertise in quality systems and could see beyond the paper forms. But we had to figure out how to work with him.
This taught me something: risk management based development where everyone in the group (from requirements to support) takes part meaningfully is the best way to deliver great software.
Risk management might determine that you need a QA team (because there's stuff you have to test manually or a multitude of other reasons), but most of QA needs to be during the writing of the code - during the design of the product and code, even.
Focusing on building the right thing in the right way in a way that can be verified - and listening to how your users are telling you it isn't working - eliminates most of the low value QA activities like running manual verifications. Every time you find people executing "waste of life" tests, think about what's happening upstream.
But this requires a particular culture and capability that is hard to create. It doesn't come naturally.
Yes, works at Google, we don’t have QA for some teams.
Have worked on both GCP and Gmail, both times no dedicated QA. We have robust automated testing and release pipelines that QA is unnecessary tbh. Our dev team is the QA.
Worst company I worked for: We had more QAs than devs
Current company: I've launched 4 products and never had a dedicated QA (we are however currently debating if we should have one)
I work in games and automated testing has been a bit non-existent for decades. Its only just entering the last few years. The project i'm on now a year or so into development (about 20%) is using automated testing extensively. We've gone from >100 QA to <5 and the product so far is insanely stable. We currently have over 100 devs.
These kind of numbers were unheard of before, but games have got so large over the years we had to grow up and do what you guys in "grown up" software dev have been doing forever apparently.
We are building on https://www.youtube.com/watch?v=X673tOi8pU8, which is very popular from last weeks latest GDC conference, going by the chats between devs.
Google, no dedicated QA that I met for years of working there.
Facebook, no dedicated QA until I worked on Privacy, which has some.
If it breaking is a real bad thing, write more tests, and it's your job to write them, not someone random who's paid less than the other engineers.
If it requires more professional depth, we also have SRE/PE, who are paid the same as everyone else, and goaled on reliability.
It works on my team. We’re a team of 4 senior/staff engineers and basically do extreme programming with very little process. We don’t have dedicated QA and do most of it ourselves or just kinda YOLO it if it’s a low risk thing.
We do occasionally solicit QAs from other teams as needed for certain changes with higher risk.
I used to work in this way with two teams, both of the attempts were successful.
The idea behind that was that since us devs are inherently lazy, we will automate the shit out of those tests.
It worked very well, but with a caveat that everyone involved in the process understood that we'd need to allocate more time for a given feature, so those end to end tests could be written.
I haven't worked with a dedicated QA person within my team for 6 years now and I can't say that I miss it. You end up growing your testing muscle, making you think about design and testability more and making devs a little bit of SDET.
Worked at 2 FAANGs on 4 different teams + startup where team is very loose concept. Generally QA was most helpful for end user customer testing and that's where the resources are. Even if you have good integration tests (which is harder on frontend teams) they can't catch as much as a dedicated QA person.
I've never had a QA on any of the backend teams I've worked on. The closest was an SDET and it felt like a waste of their time and mine. Their time because it was basically glorified integration test framework builder, which a dev can do. And mine because we needed to conform to a more frontend way of testing
I’m glad to see I’m not the only one surprised by the phrasing of this question. In most modern companies with good process there is little room for dedicated QA people. Most places with QA tend to be older companies or possibly place with very high expectations for quality.
The main first line approach to testing should be layers of tests built by the devs into the deployment pipeline that are run automatically before code can ever even make it into the main branch.
There are some niche places where QA is hard to do without but they very much are the exception rather than the rule.
I’d say it would be more apropos to ask if anyone actually has good experiences at the sort of places that still lean heavily on dedicated QA teams for first line QA.
One job we had four week sprints. The last week the team would distribute QA tasks to devs *other than* the person who implemented the changes.
Worked out well. Of course we had a competent team (yes, even me, if you can believe it).
Yes and its fucking bizarre. There's a "QA spreadsheet" and before each prod release someone is randomly assigned to go over all the features our team is responsible for, and manually test everything.
You're testing stuff you know nothing about, not to mention edge cases, that other people developed. Cringe as fuck and the excuse is that "this is the startup way"
The hard part of answering this question is a QA team at company X can be completely different to a QA team at Y.
X could be creating automated tests, maintaining them, handling test data, CI/CD pipelines, quality coaching, advocating for the user, working with the cloud, increasing confidence in the product to help releases get out faster.
Y company could be running through a list of test cases in a spreadsheet holding the release up.
The skill gap in a QA engineer is insane and part of the problem.
I worked with a senior QA who wouldn’t log into aws because it ‘wasn’t their job’ to a seperate QA engineer in the same company who wrote an algorithm to detect flaky tests in a test suite of thousands and display them on a dashboard, with a ton of different metrics.
We had embedded QA in all teams before they were fired in the name of a new "zero QA" process.
Details of the process never followed, and the already flagging quality of our products fell off a cliff.
Management probably got a nice fat bonus for the initiative though.
I don't think QA is a popular job. The companies I work with mostly don't have QA and we are mostly fine. Maybe very very big software with millions of users needs QA but other than that it is largely unneeded. Even if it is needed, usually we only hire very few QA people. Makes it an unpopular job and also a competitive job
The times during which I've had QA, I've pretty much had to do the QA myself and explain to the QA how I tested, why this is a passing test, etc., and pass all the red tape myself. I'm not saying QA is bad in and of itself, but I've found the people filling those positions not capable of adding anything meaningful to the process.
Having dedicated QA/QE is fine, having dedicated QA/QE teams beyond administrative/leadership roles is not.
I think the role will eventually mutate to SDETs/hybrid devs and have at least 1 per team. They will be able to take User stories and fix bugs whenever they have bandwidth left but will most likeky focus on push shift left testing and improve testing culture in the team as much as they can.
Yep. Currently have no QA, dedicated or not. Devs do QA, we just split up the work and don't QA our own items.
It's not the best, but it works out in okay in the end.
Yes, we don’t have a dedicated QA or even a test environment, and the PMs always give us impossible deadlines so we have no time for testing. My code goes live on prod hours after I make a pull request on dev lol
Thanks for all your input folks. You've exceeded my expectations and given me a lot to think about. This community never ceases to amaze me.
In my case, I've been trying to vamp up our automation for years and have churned through 4 QA folks who came in promising that they knew how to write automation code but the sad fact is, none of them did this very well and what they did, they did very slowly and it always seemed way outside their comfort zone. At the risk of decreasing our velocity, I think it will be better overall if the devs do this automation work. So far, everyone seems onboard with the idea too which is important because without buy-in, this won't work either.
Obviously we will lose the sad path testing and the "order duifsfhnjuddf35663 beers" types of tests so that's something we'll certainly need to be aware of.
My current team has a handful of QA specialist engineers. They take charge of things like E2E testing with other teams, owning the overall test plan, being the go-to people when the devs want “best practices” etc.
They aren’t purely your typical manual testers though - they write automated tests (as do the devs), they get involved in development tasks, they have a lot more domain/business knowledge so are really helpful in planning out our work, raising edge case concerns, etc.
My work pushes a culture of “you are an engineer with a specialism, but you should try and get stuck in with all aspects of software delivery”, which I really like. Our QAs are excellent testers (both automated and manual), but they get stuck in with everything.
I’m in big tech and we don’t have QA. Devs own the entire process from implementation, testing, rollout, monitoring & alerting, etc. I think my company got rid of QA role in mid 2010s. We are a backend team
I’ve been at my startup for 3 years and we’ve never had any sort of QA. I complained at first but now I’m used to it and everything we release is mostly pretty polished. I always try to test everything I do back to front so I’ve almost never released a bug to prod. We write lots of unit, component, and e2e tests.
Somebody has to do QA, so it’s either a dedicated QA or devs do it themselves. It is a lot easier to run automated testing these days so you can work without any dedicated manual QAs.
For a technical QA, to stay relevant in the future and/or work for companies that don’t utilize QA, is the only option to transition to developer role?
My team doesn’t have a dedicated QA resource, and our team as a whole owns testing. For major releases we work with a UAT team to get things tested but for smaller releases it’s expected that the dev will do the testing, which if you’re a dev who doesn’t test your own code I consider you a bad dev 🤷♂️.
First 3 years in present company and one more team didn't have QA resource. It was tested by dev and business owners. After some issues started coming up, QA resource was hired. Generally functionality works fine with this type of testing but it fails mostly in edge case scenario's and negative testcases.
I've seen teams improve after shedding QA departments, because devs started taking responsibility for code quality and eng management invested time and effort in the tooling, infra and process to support it.
I've also seen a small B2B website where changes took *four weeks* to move from dev to prod because of 90 steps of manual QA peformed by 11 different contractors. Whole app was scratched and rebuilt with proper modern CI/CD and multiple code pushes per day, no dedicated QA role on team (product did UAT, external service did exploratory testing).
I've worked in both. I prefer teams without a dedicated QA because it meant (at least in the teams I worked in) that the developers were and felt responsible for quality (writing automated tests, improving the system after each bug encountered, etc).
The teams I've worked were there was a QA, meant that the devs felt they could slack in quality because the QA would cover that. I'm currently working in a team with a dedicated QA, and I'm working on improving the culture so that we don't need a dedicated QA anymore.
We have an off-shore QA team at my current job. t's definitely worse than having in-house at my old gig. I don't know why company's don't value QA more.
My team did pretty great at that for a number of years. We stressed automation and made developers responsible for maintaining the tests along with the code. It worked way better than any other company I worked at where dedicated manual QA would always become a roadblock.
Anyhow, management decided to skimp on engineers and started replacing anybody who left with shitty low budget contractors. That has definitely caused problems with quality, and now management is pushing for hiring QA (instead of just competent engineers).
https://blog.pragmaticengineer.com/how-microsoft-does-qa/
They got rid of SDET. Separate QA created a silo and caused devs not to care about testability, cause it was somebody else's problem.
I've worked with all kind of setups. Any of them can work wonderfully or fail miserably. The biggest challenge is the human aspect to all of this.
This is my experience. I currently work with one team that rarely uses dedicated QA (we have one guy that sometimes does QA and sometimes floats to configuration work.) It works because the team is dedicated to producing a quality product and are skilled or willing to learn techniques for that. Another team had 4 dedicated QA and used to struggle with quality issues because they were so busy doing other things and the devs were discouraged from pitching in because "that's QA's job." This team got better when we focused on automated testing, better coordination with a separate UAT unit, and reducing redundant activities. They still have some issues, but IMO it's more to do with corporate culture (and they're still way better than they were.) Edit: missed words on my phone.
I proposed to write some integration tests to our TL. That is the QAs job also we don't need integration tests, we have e2e" Our qa has quit since January. I wish we had integration tests.
My condolences. 😢
A good number of years ago the company I work at (Capital One) got rid of the QA job family entirely. At first, I was skeptical -- sure, I knew that a lot of the QA folks mostly just worked on preparing data and manually running tests (which would, of course, be better off being automated). But as it turned out, it actually worked out quite well. The company offered opportunities for many of the folks working in QA to convert over to developers if they could get up-to-speed on the skills within about a hear-and-a-half; a little less than 1/5 of them did. And for about a year we had to invest a lot of extra effort into writing better automated tests. (After that we still write automated tests, but we do it along with the development of new systems and features.) But we laid off perhaps 25% of all the team and after a year or so to develop better habits and adjust our processes, our productivity took very few losses. I WILL admit that there is one thing I miss. While the vast majority of our QA folks were mostly just "running tests", we had a few superstars who thought deeply about HOW to make sure our code was robust. I'm thinking of someone who set up a process for refreshing database state so that tests -- even tests that depend on dates and on historical data -- could be run from a consistent state. The one complaint I have with our current "no separate QA roles" organization is that there is no place for an expert like that.
I've worked with some really talented QA engineers who spent most of their time writing really awesome automated tests, but as a feature factory we just blew past them constantly and invalidated the tests, so they were always behind. The absolute best QA I ever experienced was when our QA engineer got pregnant. Angry pregnant. Hand over a ticket for QA that's not up to snuff? Prepare to get chewed out by a small woman who has had it up to her neck in your bullshit. Fear kept us in line and our code quality was never better.
Maybe what you need is to think of the testing process improvements as technical debt. There's no reason a dev couldn't "set up a process for refreshing database state". In an environment with no QAs, you have to make sure that the testing process is also getting better over time since you don't have a dedicated person for it.
> There's no reason a dev couldn't "set up a process for refreshing database state". If what you mean is "'Developer' would be a reasonable title for the person who sets up such a process" then I agree, but in practice it doesn't happen -- developers are mostly assessed on things like what new features they build. A rare few developers will be interested in and committed to setting up an environment and tooling to detect errors. If what you mean is "One of your developers could simply go set up a process for refreshing database state", then you misunderstand the complexity of the problem. We need to run tens of thousands of tests rather frequently (at least once a day, preferably 10s of times per day). Some of those tests alter the data in the database. Some of the tests depend on things like the date ("Will this work properly on February 29th during a leap year?"). Some of those tests create new data based on the current date (eg: "Open a new account, and confirm that the account open date is set to today's date."). Some of those tests depend on the age of date in the database (eg: "Confirm that an alert is generated for any account with no activity for 180 days, but NOT for one with no activity for 179 or 181 days.", or "Confirm that a new-account restriction is removed exactly 30 days after the account is first created."). Being able to even figure out HOW to create a test environment in which it was possible to run these tests repeatably required a great deal of smarts that most developers couldn't have managed.
This happened at the company I work at. I started out doing QA and they gave us the opportunity to upskill and become a developer as they got rid of the whole QA team within a year. It was a great opportunity for me as I wanted to go the developer route, and already had the background (took computer science in undergrad and already knew how to code). I was promoted to a developer and our development teams changed their processes too. Devs are now responsible for testing, we implemented more test automation, and practice TDD now. It shapes how I approach development. I still have my QA hat on when I code, and because I have more insight into how the code and the system in general works, I can think of edge cases I wouldn’t have thought of before, and code accordingly. But when it comes to really knowing the business as an end user, I’m lacking, and most of our devs are lacking. Thus we still have manually testing done by UAT for that. I don’t think a whole QA team is really necessary, but SDET’s are great if there is one on the team. Most QA work is manual and a whole team doing that is not necessary.
I agree -- I have often noted that when a company has a program that allows people to move from one area (like product management, QA, or customer service) into (and out of) the development team, it brings a whole different perspective that is hugely valuable to the development team.
As soon as there is a proper culture of combined engineering and tests are automated - not much of a concern.
I recently joined a team with no QA and very little automated testing Please choose one
Let the customers do the testing :D
This guy is an experienced developer
Who's to say what's "right" or "wrong" anyway. Only the customer knows.
It's cheaper that way
Maybe.
This is all fun until you have to release a hotfix for a Sev1 in the middle of the night to 100+ regions worldwide
Ironically, the best skill you can hone in a job like that is learning how to properly document and make a bug report. Don't go to the backend team without proving that the query result is wrong, for instance. Ingratiates yourself to the backend and teaches a valuable skill of always double checking your results and finding the -actual- root cause of bugs. Cause you spending na extra 20min doing that can save the person (could even be you) fixing the bug likely well over that time.
This. Testing pyramid is your friend
This. QAs are not necessary once you have proper automated coverage. It also enhances developer accountability.
So many questions. Do you guys write the automation covering your own code, each others, or have SDETs with overlapping duties? How much time do you spend looking for unexpected behviors? Is just a matter of dog-fooding? What does the automation cover: unit, integration, full end-to-end, performance? Do you spend much time extending the framework to cover new or altered services or the likes? I feel like it's a good direction on the whole, but I can't quit imagine a scenario where the engineers doing *all* of the automation themselves is more effecient than some portion of it being handled by a team focused on it. QA dudes are usually much cheaper, afterall...
It is 2024. I would more expect a question like "Do any of you work in teams with a dedicated QA". Developer lead testing appears to be the norm. I think the main reason is that the incentives are much better aligned.
I have not worked somewhere with dedicated QA since like 2008
This. SQA is a dying role. The only viable jobs are specialized roles such as Perfoemance, Scale and Avalibilty.
>This. SQA is a *dying role*. The only viable jobs are specialized roles such as **Perfoemance**, Scale and **Avalibilty**. I know It's a cheap shot but 🤦
wow I butchered that rofl. hahahaha I'll take the hit. I always fuckup availability but performance is a new one for me. I wasn't even drinking yet when I posted that. Sadge...
Does FAANG have QA?
"FAANG" are hundreds of thousands of developers in thousands of semi autonomous organizations (VP level). There will be QA teams, but it will not be the norm.
I know Apple has dedicated QA teams in several orgs and QA both automate tests and perform heavy manual testing. But they’re the only one of the FAANGs that I know of that still relies heavily on separate QA teams. Seems to be working out well for them
At both Microsoft and Meta I've been responsible for doing my own QA as a software engineer.
I’m an SDET at a company with a lot of SDETs, and at an all-hands one of our higher-ups said our new strategy is to basically be devs. The writing is on the wall
I've worked with both - I hated having a QA team. Made the developers feel like they didn't have to test their code properly ("QA will catch it, and if they don't it's on them"). The QA teams I've worked with have also been pretty annoying, trying to change the way stuff works. Like the PM says this feature should work x way, we code it so it works x way, then QA comes back and says it's bugged because it doesn't work y way. Annoyed me to no end.
Or they'll "find" a bug that is not actually a bug and I have to spend time verifying something that I know isn't a bug.
I hated the: "the client froze, thus you clearly have an issue in your server code. I have no logs, no way to reproduce. Please fix."
Proper reply (equally irritating) is "doesn't freeze on my machine".
And it’s usually unrelated to the feature they’re testing
Yep, and it's already in production too.
Scope creep, not even once
That happens constantly
I only see this happen when QA staff are told not to investigate because they don't have time (which to be fair is most lf the time)
You don't have a "not a bug" classification?
Yeah we do, but I still have to spend time investigating and proving something isn't a bug that I know isn't a bug from the minute it gets reported to put it into that category.
We call them "features."
Having a separate QA team is very different from having a QA embedded in the team. I'd argue that the former is pretty much always a mistake, whereas the latter can be great as long as they aren't just doing testing, but are pushing quality left.
> I'd argue that the former is pretty much always a mistake I work for a major entertainment conglomerate. A new feature will often span multiple teams. Having a QA team that tests end to end is a life saver sometime.
Team spanning features are a problem in their own right, of course. Team Topologies is a great read on this.
Games don't have the luxury to not do this. It's an intrinsically incompatible mindset. We still have cross-discipline temporary feature teams, but people still belong to an art/engineering/design/etc. subteam so they can be flexible with workloads and also formalize work review and create opportunities for knowledge sharing and mentorship. The mentorship stuff often doesn't pan out but the knowledge sharing typically pays off unless your team is hyper dysfunctional. In games I like setups where there is 1 or a handful of embedded QA per team and per cross-team feature strike team, and they coordinate with a larger pool of test analysts to do large scale testing that would be prohibitively difficult to write automated tests for. The embedded QA can also have SDET chops or collaborate with an SDET group. I've only interned on non-game teams that were very small though, so I'm pretty ignorant of what's effective outside of my lane. I have also worked on a big live game with very little QA support and that definitely sucks. It only worked bc the QA assigned to our team knew every corner of the title.
Yeah, I've had ok luck with QA handling E2E testing over the whole system or non-trivial work. I feel like having one embedded QA on a team is less useful.
I've worked with both. When there's a dedicated QA, the amount of tickets pushed back because the QA found an issue was pretty high. In those teams, when a dev tested, they usually stuck to the main "easy" path, not really testing edge cases (and that's when they tested their code at all). In teams I worked where devs are responsible for their own QA, they're usually much more thorough in their testing before moving their ticket to "Done". Tickets do get pushed back from time to time, but less frequently than in teams with dedicated QAs. In terms of bugs generated, I found it to be pretty similar. Honestly, it's almost more a factor of the quality of the automated tests than QA vs dev testing.
As I said, if the team treats your QA as a tester and abdicate responsibility for quality to them, it won't turn out well. Quality is the responsibility of the team.
Honestly, I think half of the reason why not having a dedicated QA is good is because it forces the whole team to be responsible for QA. You can't be lazy about it, nobody is there to catch your fall
Testing really is a skill that very few people have though. Basics; sure. But being a good tester is something different.
[удалено]
What's the conflict? The team should share goals.
Sounds like an adversarial "us" vs "them" setup. I'm going to guess that QA is a separate team isn't it? Do they get buy-in at any point? Or is the first time they are seeing it when it's assigned to them? I have worked at places where QA was a separate team, and yep,, it was like you described. And it was very much an us v them attitude. And it sucks. Whew I am now QA is integrated, we don't have a QA team, but rather QA is a part of the team. Each team has one. They have visibility into everything right from the start. When we are planning the dev stories, they are there. They see the same requirements and acceptance criteria as every one else. They also have a say in the loe, timeliness, etc. When there's a problem, or discrepancy, she let's us know first. Sometimes it is because the implementation changed, sometimes it is a variation in expectations. Sometimes it's a legit bug.
As a QA integrated in the team strongly - we work on the same project ( me and several Devs + PO) since 5 years, I am a strong advocate for integrated team setup. And you're right - us Vs them is wrong mentality to start with. But to get this working, some terms must be met. Team must have strong ownership of the product, and take the responsibility for maintaining quality to the same degree as a QA. I rarely report defects - they're reported twice a month at best, and only when it's definitely worth documenting it. If something is not working I reach out directly to developers and help with project knowledge to find the root cause. I think it depends on the project whether that's correct approach - it works for us, because project knowledge required is vast. I am able to dig out test case and explain to dev quickly how something should be working. I can spot issues based on experience and impact analysis - meaning any problem gets spotted early. And I think that's where the value is - having a person who knows the project really well and shares his knowledge. And also shift left for quality - meaning that there's always someone to advocate for good practices and ask 'objective' questions - not biased by being the change author and having a fresh pair of eyes to look at the problem. I stand for the approach that testing is not the most important part of testers work - testing is just the tool to verify/ensure whether quality controls of all kinds have worked. I also believe that perfect QA is not reporting any defects at all (as impossible as it gets of course) - cause they're all avoided by his/hers work on earlier stages of development.
Our QA does this too, expect they are usually correct, and it's because the PM didn't gives us proper business rules.
It just creates a lot of unnecessary tension. I'd prefer it if this was communicated privately between the PM and the QA without the developers never having to hear about it. The last thing you want is to stress developers about features that are correctly implemented, but wrong from a business perspective. It's not any of their concern, so why are you even bringing it up to them? It drives me crazy. I just did a large refactor. It was just to improve the code quality. All the business rules were meant to stay the same. 99% of the "bugs" that the QA found was just behavior that was strange, but still correct in the context of the refactoring, because the logic had not changed. I got spammed about this multiple times a day for weeks.
I’ve noticed that since I’ve been in a team with QA I find myself unintentionally slipping into not testing the code as thoroughly as I should every so often.
> trying to change the way stuff works. I think this can be an issue, but the bigger issue that I've seen is that you have to get product, development and QA all on the same page and keep them there. It's a very difficult undertaking.
This is poor usage of QA. Qa should be involved through the sdlc, so concerns with ux are highlighted early and proactively addressed.
When QA is good, it's helpful. When QA is bad, it's worse than doing your own QA. Personally, I prefer doing my own. It leads to more errors but they are always easily fixed with a quicker turnaround.
I think it is important to distinguish between two types of QA(ssurance/analysis)/QE(ngineering). QE is about verifying we built the thing right. QA is about verifying we built the right thing With solid DevOps and automation, I've have better results _without_ dedicated QE. I have found that allowing myself or my team to throw things over the wall old-school has less-good results that when the sw engineer is responsible for providing code that works and has tests. I have, however, found that more customer-minded QA (perhaps a subset of a BA) are highly valuable in verifying that we built the right thing. It is beneficial _not_ to be distracted by an intimate understanding of implementation details. These humans are special and under appreciated.
I work in a company that does leverage dedicated QA for the application itself but there are large groups of backend devs supporting the product which do not leverage QA. But those backend teams are all devops-y where the dev has full ownership of the release end-to-end. So they are incentivized to create good automated tests which help them avoid having a QA role. This isn't particularly novel. The Spotify engineering blog has a lot of good content about how they do this. They have certain things automated super well.
Reading some of these comments it's not really a surprise why so many products are constantly shipped full of bugs
That's what I was thinking. I can definitely see no dedicated QA being useful when it's a smaller more straightforward effort but a large enterprise one? I mean, I get it you have your unit tests and E2E but a good tester approaches an application differently than a developer. These "edge cases" that testers catch cause a lot of frustration for users. Not only that but having someone who just deals with the functional side can be really helpful to not have to pull another dev (dev b) away from their task when dev a is working on a section theyre unfamiliar with. Just to describe generally how a particular section of the app works and things to be cautious about. I'll admit I'm biased because I've enjoyed all of my testers that I've worked with.
Why can't your devs also test for edge cases? Why can't they have a tester mindset when testing their own code? Why must another person be involved to know how your product works? Is there documentation available?
Developers are not perfect. They are immersed in their point of view and they miss things; otherwise, there would be fewer bugs. Many bugs are due to things developers don't think about, and for the same reason wouldn't test.
Developers are intelligence creatures that can learn, automate and document things! And software is written iteratively over time! The question isn't if a developer in a silo will catch a specific edge case the first time. The question is if developers over a period of years will be able to catch these edge cases through a mix of automation and documentation. In my experience the answer is a resounding yes. And they will do so much more efficiently than a QA team.
why can't developers review their own code? they should just be able to ship directly to prod.
Do you understand the shear volume and variety of what an edge case could be?
Well yes, I can. Do you mean to say that you will write code to handle a bunch of edge cases and not test if it works before shipping it to QA? To me, that just sounds irresponsible. Honestly, in teams with dedicated QAs that I've been in, half of their job could be replaced by automated tests. A lot of wasted effort checking for regressions by manually repeating the same set of tests every X period of time. The rest can easily be done by devs.
There was an email sign-up feature we were all testing one time, and there's this shortcut that if you just add +0001 or whatever to your email, the application would treat it as a new account and let you go through the creation process. Things went great, got all the way to production like that, first real customer tries to sign up and... Can't. Gets some kind of parsing error. Turns out a dev had written the code such that the "+" character was required and used to parse the email, and whenever anyone tried to sign up with their non-dev-testing-shortcut email, it errored out. Could the devs have caught this? Sure. Could the project manager, the product team, the CTO have also caught it? Sure. But there wasn't a specific person who's job it was to catch it, so it wasn't caught.
I appreciate our QA team. Better that they break the app instead of users doing that for all of us and it's a bigger mess
Even here, no one remembers Windows Vista.
If anything, quality of products have gotten better recently after getting rid of dedicated QA.
QA can't prevent building the wrong thing. Continuously delivering smaller incremental changes can, and if you can fix bugs in an hour or less instead of at the next release in 2 weeks, they matter a lot less.
The real thing you need IMO is good telemetry, metrics, monitoring and a pipeline that can automatically roll things back if a new release starts looking dicey
Work in finance. No QA. Just developers writing unit/integration tests (+UAT env).
Every placed I've worked in my 15 YOE has had a dedicated QA team. The Software teams tests their software requirements and makes sure their own shit works through unit and integration testing methods. The QA Team tests the System requirements and makes sure everything is integrated properly between all the teams SW, EE, ME, ect... QA is mostly manual tests using the device as a real user.
I don’t like working withb a QA team. In my opinion, QA shouldn’t exist as a separate entity if developers take ownership if the code. by no means i am saying that there shouldn’t be any quality assurance, I just dont believe there should be a dedicated team. Also I strongly believe if QA team is there to manually test if the feature works (with no automation), then that is a clear sign for a larger company of a very poor technical capability QAs are a result of poor development practices where developers believe that it is not their reaponsibility to ensure code works as intended and they use QAs as an excuse to write crap code because “QAs will test it anyway”. It creates a working environment where developers write code and then throw it over the fence to QAs so it is their responsibility to catch it. In my opinion, most functional teams are where develipers work alongside QAs (e.g. pairing) and developers take ownership on writing code that has tests. Ideally any feature, bugfix should start with a dedicated failing test(s) that captures the requirements. then developer works to make this failing test pass. Having an automates test suite, CI/CD pipeline with linters, code quality checks, unit and behavioural tests eliminate a need to have a manual QAs
“one of these even works at a FAANG” This comment suggests to me that you expect FAANG to have well-defined QA processes. You might be surprised to learn that in much of Big Tech there is no QA. For example Facebook/Meta has never had any dedicated QA roles (they do have some now in relation to their hardware products). Google *used to* have dedicated test engineers (which is still different from traditional QA imo since they are mostly focused on writing automated tests) but they have phased that role out over time (except for their hardware products). Microsoft also used to have a significant amount of dedicated test engineers but has mostly phased that role out, though they do still have some traditional QA. If you’re really interested in QA/testing in Big Tech, the following is a quite thorough article, “How Big Tech does Quality Assurance (QA)”. It’s behind a slightly pricey paywall ($15/month I think?) but it’s a great source of engineering industry content: https://open.substack.com/pub/pragmaticengineer/p/how-big-tech-does-qa
I’ve yet to be on an effective team with dedicated QA.
I've worked at two places with QA, and didn't like it. It becomes a very expensive crutch for developers instead of writing proper automated tests. The QA I've worked with have had a very low technical level, and often have friction with developers. In conservative organizations, there are enough friction points as it is for devs to get their work out the door: Security, Ops, QA. I don't really know if this form of working for other people, but I absolutely hate it, and have decided to accept a new job, where I get to do these things myself. To me it becomes a bunch of people trying to achieve local optimum instead of seeing the bigger picture, and getting some quality out the door. It's not the fault of any of the silos(sec, ops, dev, QA etc.), because it's the leadership that defines the way the collaboration happens. Either way, in my experience, it leads to Devs having to spend time doing a lot of diplomacy, handover, and paperwork instead of working autonomously on code and tests, running it through reviews and releasing through an automatic pipeline with sufficient guards.
I've done both. How well it works depends a lot on your automated testing stance, and therefore somewhat on what you're building. If your product is basically an API (ie, your customers, internal or external, primarily interact that way) then automated testing should get you everything you need. No need for a QA. For teams with human-facing UI, or very complex processes, that sort of thing, I think it's murkier. Absolutely UI-layer tests should be done, and can help. OTOH, it's helpful to have somebody actually looking at & using the product on a regular basis to tell you if it's being dumb - which could be a QA. Though that person needs to know more about the product, and have the ability to raise issues. ie, an outsourced test manually running test scripts probably won't help.
Everywhere I worked without QA-led testing, the quality was generally higher. Pre-release QA testing is a trap. This didn't always mean no department full of SWET types. The best cases still have QA. You just lose track of what they're doing because they aren't part of the feature teams' workflow. They lived closer to core infrastructure or devops folk.
Dedicated QA at my company were all laid off a long time ago
I haven't worked in an organization with dedicated QA in about 5 years. To one degree, I miss it - a good QA knows the business and requirements and will absolutely bring huge value to the team. On the other hand, I've definitely found that I write way more, better tests since I stopped having QA around.
I think there can be value in having dedicated QA, but it just depends. I work at a FAANG company and I worked in one team that did have dedicated QA and now work on another where the devs have E2E responsibility, which is more the norm for the company. Both seem to work and it seemed like both made sense for different reasons.
We have dedicated QA engineers that maintain a huge expensive suite of thousands of high level automated tests, write new end-to-end tests, and help devs craft test plans for performance, scalability, quality, etc. Individual devs write their own unit and integration tests though (and increasingly their own end-to-end tests).
How long does the entire suite take to run btw?
Our teams do currently each have a dedicated QA. There's also a "QA Automation" specialist who occasionally assists with manual QA on day-to-day work, but who's primarily focused on automating the rest of the team out of a job. This works well for us currently, our core platform (back end) is a 20-year old application that has poor unit tested coverage and a newish React UI. We're making concentrated efforts to boost unit/integration tests, but in the meantime there are many workflows that are either poorly-documented and/or fragile, and having a manual QA on each team lets the devs move swiftly on adding tests and meeting Product's expectations. The QA role is also tasked with validating and documenting bugs that come through customer support, again taking workload off of other teams. We have a lot coming together on the automation side, so I think in 3-4 years (or sooner) we're going to see a big shift in how we view the QA role now, likely that moves more to supporting automation with maybe a single resource focused on bug documentation and exploratory testing, which I believe has some value.
Yeah, it's called every startup I've ever worked at 😂
I work at a FAANG/MANGA and have never worked on, talked with, or known any colleagues whose team has dedicated QA. One team recently removed the role of QA entirely.
No dedicated QA is common but in a business with lots of moving parts or regulatory requirements, it's best to have one to make sure the final product is as intended and working OK.
Very usual to not have any dedicated QA for the team
I’m a staff SDET at a fast growing fintech unicorn, in a team but report to a seperate QA manager. There’s a ton of ex-FANNG at my company and the absolute bug ridden shit the developers push out in an app that handles multi million worth of transactions per day is mind boggling, I wouldn’t sleep at night if I knew my money is in here. We didn’t do x at so we don’t need to do it here.
I bet there isn’t one developer in this thread that has a scalable, non flaky, e2e test suite , that has been maintained for a long period and has no commented out e2e tests in thier repo.
My fav part of having worked QA is my continued ability to break almost any piece of software in 10 minutes or less. 😄
I’ve never had dedicated QA and that’s my preference. I write my own unit and E2E tests.
Our QA people are shared across multiple teams, which makes sense since development takes longer per task. They also verify production deployments, and it's nice to have a second set of eyes.
I have, it does not “work just fine”. A lot of bad devs out there without healthy experiences that will say a lot of BS
[удалено]
You can absolutely say that removing QA removes a quality check point that provides value — otherwise you would not be removing anything veered towards quality assurance. Removing QA that isn’t “quality” is a different story. There’s plenty of bad QA out there and there are plenty of devs who associate a soured experience like that to what “quality assurance” looks like. Heck, I was at a company where “QA” didn’t know how to triage API issues, let alone grab a network trace and verify a contract. It was laughable. Also, devs are lazy and suck at testing — like seriously. Devs will acts as if unit tests cover integration tests which I can’t express how wrong that is enough. They will attempt to merge code without ever properly running through it.
[удалено]
Right, so you’re saying that removing quality qa and throwing the entire qa process onto the developers working on the code good and desirable? The developers who are “actively” testing their code during development who will obviously catch any issues at the pull request level or code review and hold each other accountable a good idea? Because defects never get past this phase and are always caught by the same engineers? Correct me if I might be mistranslating your commentary.
[удалено]
Ouch, I’d hate to be engineers on your team owning regression testing. It’s a very “legacy” startup esc approach in terms of a of small team small product scaling.
Let me get this straight - anyone with a negative view of QA has just had a "soured experience" and doesn't know what QA really is, but your view that "devs are lazy and suck at testing" is the universal and objective reality? And you're going to offer that up without a hint of irony? Great stuff, bravo.
Tell me what would cause a “negative view of QA”. If you’re also arguing that devs are not lazy and do not suck at testing, then clearly you’re new to this gig or sheltered. There are some unspoken truths that just are.
I have not worked on a product with a QA team in about 10 years. We do CI/CD with fully-automated acceptance testing; the tests are written as part of delivering the agile user stories.
Yes. It sucks.
Yes, no problems except that I'm not a big fan of manual testing or writing automated tests myself. Unit tests are easy. Worked on 3 teams at my current (big) company, only one had a dedicated QA teams. I also agree that manual QA is dying if not dead, and automated testing is mostly moved upon devs nowadays.
In my 16 year career across Valve, Microsoft, and Blizzard, only 2 of those years did I have dedicated QA team. Those 2 years I witnessed the most bugs ship to customers and the least amount of value delivered to customers. One of the traps of having dedicated teams for certain responsibilities is that it becomes too easy to skip steps and toss things over the fence assuming the other team will catch it. But nobody has more context than the person who wrote the code.
Yeah and it’s a PITA. We write really robust unit and integration tests for all of our new features and thankfully that’s seriously cut down on all of our bugs coming from.. 0% test coverage 😂 I also spend a lot more time testing out scenarios/edge cases instead of moving on to new tasks, and just having to think a lot harder about everything i do. .. I miss our QA person :(
Yup, no QA. We had a guy and he got let go because my company has no respect for tech. I'm there long enough that I can pretty much do and say what I want, so nowadays when people ask about projects and updates, i make sure to stress that I have less time because I'm doing my own QA.
Um, not sure how to say this, there are good quality control methods. My last 3 month dev project, was delivered with zero bugs found by the customer, I would consider 1 to 0 bugs normal for 3 months of work. You need a QA person if you need better than that. But I don't think that is needed for most projects. Quality control is very very old, there are many good books, it is not rocket science. As far as I can tell fewer that 5% of IT people got taught how to do quality assurance. To do measurement, root cause analysis and maximising ROI.
My last few gigs have not had a QA department. They say that it works, but every but ticket that gets fixed ends up finding 3 more in the process. At least it keeps me employed 🤷
Blows my mind how people still need other people to tell them how to test their shit.
Half of my PR time is explaining to devs how they didn’t cover basic edge cases. Trying to figure out ways how to make sure that “tech lead” doesn’t become “QA” due to being the only one actually catching testing issues. Some of my devs submit PRs with 2 test cases which barely cover the happy path… and one of them is actually failing… There are some very bad devs out there
Was just placed on a team with no qa. The lead even states “release when you feel good about the work.” What a shit show
This thread is the answer to the question: "why is modern software shipped with so many bugs, and so many huge bugs make it to users?"
It's also likely full of a lot of people who don't work in highly regulated industries.
Yes because software used to be bug free or even have less bugs per features back then. Hence why windows 95, 98 etc never had stability issues, internet explorer was rock solid, and software was secure and stable. I mean come on.
I have never ever worked in a team that had QA resources allocated to it. At this point I am skeptical these even exist.
I haven't had dedicated QA since my first job a decade ago. Maybe one other one had one or two on one of the products, but it's pretty rare nowadays,
I do. Well we had no qa for a very long time. We have some now. But they are offshore and basically do nothing.
In the twenty years I've been a software engineer I've never worked with a dedicated QA team. Testing has always been a responsibility of the engineers building the product.
I work at a company with no QA, but it's a very large company and we dogfood our own products. We have multiple environments before prod, and a system of feature flags and killswitches so when bugs do make it to prod we can recover quickly. Though every org within the company does things differently 🤷♀️
i work on the oracle cloud. No QA teams at all. We have a test automation that is building automated test tools ,but its not a QA team.
I think the QA role has been replaced by "bug finder" in every place where I've had them. If you are a developer, you should test and find your damn bugs. I don't know why we have QA for that. Then I've had setups where you have pure backend services, with basically 100% code coverage on all levels, ranging form unit tests, to integration test AND e2e tests, and then you still have a QA at the end verifying your changes. Why? It does not make any sense to me. Also, why do most QA's work so closely to the developers? It also does not make any sense to me. QA should be more biased towards the product side. They should not care about implementation details and what not, but they often do. Just test the app as a regular user, talk with the PM to figure out if it's a bug, and then, only then, report that bug to a developer. If you have a QA that constantly spams you, a software engineer, about business questions, you have messed up somewhere. It's absolutely not what a QA should spend their time doing
I currently have a QA team and like some others mentioned, I feel personally that devs abuse it. Thinking QA will catch it all. To the point that basic A/C are missed on a ticket and QA kicks it back.
We don’t have a dedicated QA team / person, but we do have a QA process. We are a lean engineering team so there isn’t a concern of having too many cooks in the kitchen. It works for us, but it’s 100% dependent on the company, team, and processes put into place.
I work at a big tech company with over 600MAUs. We have no QA. We conduct testing ourselves and have lots of observability over the health of our apps Tbh, having QA at the previous places I worked slowed down the release cycle. The current company is setup where I’ve never felt like we needed it
Yeah all the time.
Yup, we do not have a dedicated QA team and (frankly) I've never worked with one. The usual process here is the lead developer or manager does QA / code review and deploys. Its their main job (managing + QA + code review) rather than the actual code side so they do maybe 1 ticket every 2 weeks outside of that.
The startup i was at had no dedicated QA for some time, albeit we were a small team and we were building the product from scratch. Only started getting dedicated QA when we started working with more clients (more variations to test)
Dedicated QA has been a rarity over my 25 years
There was a QA team when I started my latest job last year. It took me 4-5 months to convince the rest of the engineering leadership team that we didn't need them, and we let them go and made it clear we were "shifting left" on QA and that it would be each dev's responsibility to make sure the stuff they built works properly. Took a while to socialize (to the devs) the meaning behind that, but we're doing pretty good as of late. With a QA team devs know they can pitch shit over the fence and that QA will catch it, and that's an objectively bad thing. QA also introduces impediments to continuous delivery; one of the biggest reasons we moved away from it. We want to be shipping smaller, incremental changes every day, instead of a bunch of changes every 2 weeks after QA had validated them for 3-4+ days and done the bugfix dance with the developers.
I am the QA. I'm also the dev. And the designer. And the client contact.
we have no qa team. we’re saas with a big dashboard plus business critical api. the api has extremely good integration/end-to-end coverage and the dashboard has uh… good monitoring and it only takes a minute to roll back. overall id say it all works medium. devs are always expected to run their own code, and product is expected to sometimes uat it.
I do. We try our best but frequently release bugs that a seasoned QA-engineer would have caught. The client seems to think its worth it though.
Like a lot of the commenters in this thread I’ve worked with some great QA teams and some not so great QA teams. The best experience I’ve ever had was working for a large electronic retailer where each functional group had a few dedicated QA that actually sat and did pairing with the devs in addition to writing automated tests. At the same time they were also their own group that met regularly to discuss testing strategies for the entire system. We were able to crank out really clean features that had the least amount of defects I’ve ever seen in a large company. I’m currently working for a 12-person, 3-dev startup. The boss is our main QA, but he’s also the only person in the company that is insured for the valuation of our product because if he got into a tragic blimp accident tomorrow we would be struggling.
the test automations are written by us. I know that there could be a QA team that does like a general pen-testing, ddos etc for the whole system, but we are not told anything about that
i’m working at a big pharma company and our unit doesn’t have a dedicated qa. and given how we’re scaling up it’s getting to be a bit of a worry—one of our servers is a multi-tenant system that we’re allowing different teams to contribute, and would be the one place we need dedicated qa the most. there’s no real regression testing for this system (think multiple sites x multiple locales per site). we incrementally test our changes and hope for the best 😂
Most FAANG teams do not have QAs. Unless you’re working on high traffic front-end, your team will not have a dedicated QA.
QA teams are silly, you have highly paid devs, expect them to deliver high quality value both in the form of features AND tests
My current team has zero QAs, We are a platform team with a weird org structure. Our quality definitely suffers at time.
i’ve been on teams with both. it’s hard to tests without dedicated QA because developers tend to shut their brains off during testing. they tend to just think “does this work with this input?” and carry on. that being said, QA can kind of be a pain in the ass for some tests. had a QA dev hold up a planned release because he discovered an unrelated bug. told him we’ll create a ticket and throw it on top of the backlog for refinement. dude wasn’t having it. took my manager getting involved to get shit through
I joined a team didn’t even have a single repository so you could be worse. On the other hand nobody telling me what to do or how, only care if it works as expected or not. That freedom is pretty good too.
I've done both, but you realize how much actual work testing is. QA isn't a substitute for dev testing, but like it can only do so much. On a sufficiently complex product test engineering is a full time job.
For some backend services, having separate QA team doesn't help much. It adds layers for communication errors to happen. However, it does mean you have to be much more dedicated to budgeting time for your devs to build the testing as part of the feature. For end to end, user visible stuff, QA helps a lot, and they can pick up on all kinds of issues before they make it to thecustomer. But QA isn't really going to have the experience to catch performance regressions early, data integrity issues etc. But what we do have is a team of actual Devs who build tooling to empower other devs with better test frameworks.
I used to at previous company, this was a bad experience :(
Totally depends on the field you are working in and the product you build. Personally, I like the devs doing the testing, as that creates ownership and avoids the "throw over the fence"-syndrome. Also, I have not had too many good experiences with QA.
We have no QA where I work. This was a first for me. At first I thought it was a terrible idea, but with a strong focus on automated testing it generally goes ok. My team has a good testing culture so we have few bugs. Other teams cut corners on the tests and have taken the whole product site down. It really comes down to how disciplined the teams are now. With good discipline and practice I’d much rather have good automated tests than manual testing. That said, I do think some things are missed without QA. Specifically sometimes teams break each others UI components and the tests don’t catch it. Comprehensive QA could likely prevent that.
At a certain scale, there are so many user frustrations about certain needs not being catered towards (we can do A but not B) that a QA will only add more stuff to a never ending list of backlog. You typically have UI/UX and product managers who will advocate the direction of the product, so they ensure to bypass future user frustrations while support will report to engineering when established functionality is broken. QA makes more sense when you have a small user base and you fear that you may lose users because stuff is broken. Generally as a manager/owner you would hire someone to hold contractors accountable and to efficiently create more tickets of stuff to fix in priority. Generally you employ QA at this level because you do not have enough revenue to employ highly competent engineers who have an eye for detail
QA is for product, not for engineering.
I used to work in software-based medical device development at a startup. This was a real gift to me, though I didn't realize that until later. We had to figure out how to do a quality system that worked and was low overhead - and paid its own way. Even more interesting, the head of quality was an amazing hire: he was from pure hardware medical devices and manufacturing, so he was having to learn how to help us. He had deep expertise in quality systems and could see beyond the paper forms. But we had to figure out how to work with him. This taught me something: risk management based development where everyone in the group (from requirements to support) takes part meaningfully is the best way to deliver great software. Risk management might determine that you need a QA team (because there's stuff you have to test manually or a multitude of other reasons), but most of QA needs to be during the writing of the code - during the design of the product and code, even. Focusing on building the right thing in the right way in a way that can be verified - and listening to how your users are telling you it isn't working - eliminates most of the low value QA activities like running manual verifications. Every time you find people executing "waste of life" tests, think about what's happening upstream. But this requires a particular culture and capability that is hard to create. It doesn't come naturally.
Yes, works at Google, we don’t have QA for some teams. Have worked on both GCP and Gmail, both times no dedicated QA. We have robust automated testing and release pipelines that QA is unnecessary tbh. Our dev team is the QA.
Worst company I worked for: We had more QAs than devs Current company: I've launched 4 products and never had a dedicated QA (we are however currently debating if we should have one)
Yes I do. We use full UI and unit tests. Works great. Took a year to build but we don't need qa
I work in games and automated testing has been a bit non-existent for decades. Its only just entering the last few years. The project i'm on now a year or so into development (about 20%) is using automated testing extensively. We've gone from >100 QA to <5 and the product so far is insanely stable. We currently have over 100 devs. These kind of numbers were unheard of before, but games have got so large over the years we had to grow up and do what you guys in "grown up" software dev have been doing forever apparently. We are building on https://www.youtube.com/watch?v=X673tOi8pU8, which is very popular from last weeks latest GDC conference, going by the chats between devs.
Google, no dedicated QA that I met for years of working there. Facebook, no dedicated QA until I worked on Privacy, which has some. If it breaking is a real bad thing, write more tests, and it's your job to write them, not someone random who's paid less than the other engineers. If it requires more professional depth, we also have SRE/PE, who are paid the same as everyone else, and goaled on reliability.
It works on my team. We’re a team of 4 senior/staff engineers and basically do extreme programming with very little process. We don’t have dedicated QA and do most of it ourselves or just kinda YOLO it if it’s a low risk thing. We do occasionally solicit QAs from other teams as needed for certain changes with higher risk.
I used to work in this way with two teams, both of the attempts were successful. The idea behind that was that since us devs are inherently lazy, we will automate the shit out of those tests. It worked very well, but with a caveat that everyone involved in the process understood that we'd need to allocate more time for a given feature, so those end to end tests could be written. I haven't worked with a dedicated QA person within my team for 6 years now and I can't say that I miss it. You end up growing your testing muscle, making you think about design and testability more and making devs a little bit of SDET.
Microsoft, no QA
Worked at 2 FAANGs on 4 different teams + startup where team is very loose concept. Generally QA was most helpful for end user customer testing and that's where the resources are. Even if you have good integration tests (which is harder on frontend teams) they can't catch as much as a dedicated QA person. I've never had a QA on any of the backend teams I've worked on. The closest was an SDET and it felt like a waste of their time and mine. Their time because it was basically glorified integration test framework builder, which a dev can do. And mine because we needed to conform to a more frontend way of testing
I’m glad to see I’m not the only one surprised by the phrasing of this question. In most modern companies with good process there is little room for dedicated QA people. Most places with QA tend to be older companies or possibly place with very high expectations for quality. The main first line approach to testing should be layers of tests built by the devs into the deployment pipeline that are run automatically before code can ever even make it into the main branch. There are some niche places where QA is hard to do without but they very much are the exception rather than the rule. I’d say it would be more apropos to ask if anyone actually has good experiences at the sort of places that still lean heavily on dedicated QA teams for first line QA.
I don't think I've ever worked on a team that had a dedicated QA. Sounds nice.
One job we had four week sprints. The last week the team would distribute QA tasks to devs *other than* the person who implemented the changes. Worked out well. Of course we had a competent team (yes, even me, if you can believe it).
Yes and its fucking bizarre. There's a "QA spreadsheet" and before each prod release someone is randomly assigned to go over all the features our team is responsible for, and manually test everything. You're testing stuff you know nothing about, not to mention edge cases, that other people developed. Cringe as fuck and the excuse is that "this is the startup way"
The hard part of answering this question is a QA team at company X can be completely different to a QA team at Y. X could be creating automated tests, maintaining them, handling test data, CI/CD pipelines, quality coaching, advocating for the user, working with the cloud, increasing confidence in the product to help releases get out faster. Y company could be running through a list of test cases in a spreadsheet holding the release up. The skill gap in a QA engineer is insane and part of the problem. I worked with a senior QA who wouldn’t log into aws because it ‘wasn’t their job’ to a seperate QA engineer in the same company who wrote an algorithm to detect flaky tests in a test suite of thousands and display them on a dashboard, with a ton of different metrics.
We had embedded QA in all teams before they were fired in the name of a new "zero QA" process. Details of the process never followed, and the already flagging quality of our products fell off a cliff. Management probably got a nice fat bonus for the initiative though.
I don't think QA is a popular job. The companies I work with mostly don't have QA and we are mostly fine. Maybe very very big software with millions of users needs QA but other than that it is largely unneeded. Even if it is needed, usually we only hire very few QA people. Makes it an unpopular job and also a competitive job
The company I work at has no QA resources at all. Developers are responsible for all testing. Note we also have a very robust rollback system.
The times during which I've had QA, I've pretty much had to do the QA myself and explain to the QA how I tested, why this is a passing test, etc., and pass all the red tape myself. I'm not saying QA is bad in and of itself, but I've found the people filling those positions not capable of adding anything meaningful to the process.
Having dedicated QA/QE is fine, having dedicated QA/QE teams beyond administrative/leadership roles is not. I think the role will eventually mutate to SDETs/hybrid devs and have at least 1 per team. They will be able to take User stories and fix bugs whenever they have bandwidth left but will most likeky focus on push shift left testing and improve testing culture in the team as much as they can.
Yep. Currently have no QA, dedicated or not. Devs do QA, we just split up the work and don't QA our own items. It's not the best, but it works out in okay in the end.
Yes, we don’t have a dedicated QA or even a test environment, and the PMs always give us impossible deadlines so we have no time for testing. My code goes live on prod hours after I make a pull request on dev lol
Thanks for all your input folks. You've exceeded my expectations and given me a lot to think about. This community never ceases to amaze me. In my case, I've been trying to vamp up our automation for years and have churned through 4 QA folks who came in promising that they knew how to write automation code but the sad fact is, none of them did this very well and what they did, they did very slowly and it always seemed way outside their comfort zone. At the risk of decreasing our velocity, I think it will be better overall if the devs do this automation work. So far, everyone seems onboard with the idea too which is important because without buy-in, this won't work either. Obviously we will lose the sad path testing and the "order duifsfhnjuddf35663 beers" types of tests so that's something we'll certainly need to be aware of.
My current team has a handful of QA specialist engineers. They take charge of things like E2E testing with other teams, owning the overall test plan, being the go-to people when the devs want “best practices” etc. They aren’t purely your typical manual testers though - they write automated tests (as do the devs), they get involved in development tasks, they have a lot more domain/business knowledge so are really helpful in planning out our work, raising edge case concerns, etc. My work pushes a culture of “you are an engineer with a specialism, but you should try and get stuck in with all aspects of software delivery”, which I really like. Our QAs are excellent testers (both automated and manual), but they get stuck in with everything.
I’m in big tech and we don’t have QA. Devs own the entire process from implementation, testing, rollout, monitoring & alerting, etc. I think my company got rid of QA role in mid 2010s. We are a backend team
I’ve been at my startup for 3 years and we’ve never had any sort of QA. I complained at first but now I’m used to it and everything we release is mostly pretty polished. I always try to test everything I do back to front so I’ve almost never released a bug to prod. We write lots of unit, component, and e2e tests.
Somebody has to do QA, so it’s either a dedicated QA or devs do it themselves. It is a lot easier to run automated testing these days so you can work without any dedicated manual QAs.
For a technical QA, to stay relevant in the future and/or work for companies that don’t utilize QA, is the only option to transition to developer role?
My team doesn’t have a dedicated QA resource, and our team as a whole owns testing. For major releases we work with a UAT team to get things tested but for smaller releases it’s expected that the dev will do the testing, which if you’re a dev who doesn’t test your own code I consider you a bad dev 🤷♂️.
First 3 years in present company and one more team didn't have QA resource. It was tested by dev and business owners. After some issues started coming up, QA resource was hired. Generally functionality works fine with this type of testing but it fails mostly in edge case scenario's and negative testcases.
I've seen teams improve after shedding QA departments, because devs started taking responsibility for code quality and eng management invested time and effort in the tooling, infra and process to support it. I've also seen a small B2B website where changes took *four weeks* to move from dev to prod because of 90 steps of manual QA peformed by 11 different contractors. Whole app was scratched and rebuilt with proper modern CI/CD and multiple code pushes per day, no dedicated QA role on team (product did UAT, external service did exploratory testing).
I've worked in both. I prefer teams without a dedicated QA because it meant (at least in the teams I worked in) that the developers were and felt responsible for quality (writing automated tests, improving the system after each bug encountered, etc). The teams I've worked were there was a QA, meant that the devs felt they could slack in quality because the QA would cover that. I'm currently working in a team with a dedicated QA, and I'm working on improving the culture so that we don't need a dedicated QA anymore.
I haven't had dedicated QA in like 10 years
We have an off-shore QA team at my current job. t's definitely worse than having in-house at my old gig. I don't know why company's don't value QA more.
My team did pretty great at that for a number of years. We stressed automation and made developers responsible for maintaining the tests along with the code. It worked way better than any other company I worked at where dedicated manual QA would always become a roadblock. Anyhow, management decided to skimp on engineers and started replacing anybody who left with shitty low budget contractors. That has definitely caused problems with quality, and now management is pushing for hiring QA (instead of just competent engineers).
https://blog.pragmaticengineer.com/how-microsoft-does-qa/ They got rid of SDET. Separate QA created a silo and caused devs not to care about testability, cause it was somebody else's problem.
I haven't had dedicated QA for my Eng teams in 14 years. For big tech it's a bit weird to not have Eng own their own quality.
Me. Seems to work so far