TestDino Is Useless If You Already Use Playwright
I keep seeing TestDino pop up around Playwright. The pitch sounds reasonable. You already have Playwright tests. TestDino sits on top of them and gives you: centralized reporting flaky-test detection failure grouping AI analysis historical trends screenshots and traces PR integrations an MCP server Basically: Playwright ↓ TestDino ↓ More dashboards I like testing tools. I like Playwright. I particularly like tools that remove annoying QA work. So I went into this expecting there to be something interesting here. And there is. Just not enough for me to understand why most Playwright teams would pay for it. The more I looked, the more TestDino started to feel like SaaS rent for looking at your own Playwright results. First, what does TestDino actually do? This is important because the website throws around phrases like: AI-native test observability and: cloud companion for Playwright Underneath that language, the product is much easier to explain. You install their Playwright reporter: reporter: [ [‘@testdino/playwright’, { token: process.env.TESTDINO_TOKEN }] ] Your Playwright runs get uploaded to TestDino. TestDino then stores, organizes, analyzes, and displays the results. Its own documentation describes it as a platform for Playwright test reporting, failure analysis, flaky-test detection, integrations, analytics, and historical results. That’s useful. But let’s be clear about what you’re buying. You’re not replacing Playwright. You’re not getting a new browser automation engine. You’re not getting some magical testing infrastructure that suddenly executes things Playwright couldn’t execute. You’re mostly adding a SaaS layer around Playwright. And that’s where I start having questions. Playwright already gives me a ridiculous amount of debugging information Playwright isn’t exactly starving us for evidence. A failing Playwright test can already give you: screenshots videos traces DOM snapshots network activity console output source information action timelines retries TestDino’s own trace documentation explains how its trace viewer exposes the actions, DOM, network, console, source, and timing information captured by Playwright traces. That’s nice. But I kept having this thought while looking at the product: How much am I paying TestDino to organize information that Playwright already generated? The answer isn’t “nothing.” Centralized history is genuinely useful. Cross-run flake detection is useful. Grouping failures across hundreds of tests is useful. But this is a narrower value proposition than the marketing makes it sound. If your Playwright suite is 150 tests and your team can already diagnose failures from CI artifacts, TestDino feels like bringing Salesforce into a lemonade stand. And it only works with Playwright This one is pretty brutal. TestDino is explicitly built around Playwright. If your company has: Playwright + Selenium + Cypress + mobile automation + API suites you don’t really have a central testing platform. You have a Playwright dashboard. Everything else still lives somewhere else. That may be completely fine for a young company standardized on Playwright. It’s much less appealing for a mature QA organization. The weird part is that the more sophisticated your testing organization becomes, the more likely you are to have multiple types of automation. Which makes TestDino’s narrowness increasingly noticeable. The pricing looks reasonable until you do the math At first glance, pricing actually looks pretty good. Current published pricing is approximately: Free $0 Pro $49/month Team $99/month with lower prices if billed annually. Free gets 5,000 executions. Pro gets 10,000. Team gets 40,000. Not crazy. Then you read the important part. TestDino says: One test across 3 browsers = 3 results. That’s where this gets interesting. Imagine a fairly modest suite: 500 tests × 3 browsers × 30 daily runs That’s: 45,000 test results/month Congratulations. You’ve exceeded the published Team allowance. With 500 tests. Running once per day. That’s not remotely unusual for a serious automation team. And if you’re running the suite: per PR + nightly + staging + production those numbers can climb very quickly. Usage-based pricing isn’t inherently bad. But TestDino’s value increases when you run lots of tests, while the pricing also increases with how many tests you run. That’s an awkward incentive. The user limits are weird too The Free plan allows one user. Pro allows three. Team allows 30. So suppose you’re a tiny QA team with: 2 QA engineers 2 developers You may have very low execution volume. Doesn’t matter. If all four need proper access, you’re already outside Pro’s three-user allowance. That jumps you toward Team. For what is still fundamentally a Playwright reporting dashboard. This is the kind of SaaS packaging that makes me tired. Then there’s artifact retention This part bothers me more. According to TestDino’s pricing table, Pro and Team have: 21 days artifact retention while analytics may be retained much longer. Think about what TestDino is selling. Historical context. Debugging. Failure analysis. Understanding recurring issues. Now imagine: “Didn’t we have this exact checkout failure last month?” Great question. The analytics may remember it. But the screenshot, video, or artifact you actually want may already be gone. Three weeks feels short for a product whose entire purpose is to help me understand historical test failures. I can obviously keep my artifacts elsewhere. But now we’re back to: Why am I paying another service for this? The pricing page itself confused me Here’s something I didn’t expect. On the pricing page, the Pro plan says it includes: AI classification AI test analysis as additions over Free. But lower on the same page, the FAQ says every plan includes AI failure classification. Maybe there’s a perfectly reasonable distinction between exactly which AI features are included. But I shouldn’t have to reverse engineer the pricing page to understand it. This is a test analytics company. I expect the test results to be clear. I’d also appreciate the pricing being clear. Uploading all this debugging data is not a trivial decision Remember what we’re sending. Potentially: screenshots videos traces logs DOM state network information test names branch information commit information failure data TestDino advertises ISO 27001, SOC 2 Type II, GDPR support, and EU hosting options, which is reassuring. But its pricing page lists data redaction under Enterprise features. That makes me uncomfortable. For a reporting product ingesting screenshots, traces, logs, and potentially application data, redaction isn’t some exotic Fortune 100 feature. It can be basic hygiene. Maybe your test environment contains nothing sensitive. Great. But I’d check that assumption before uploading every Playwright artifact your company produces. The AI part doesn’t move me much TestDino groups failures and categorizes issues. The product talks about classifications such as timing, environment, network, assertion, bugs, UI changes, and flaky behavior. That sounds useful. But I don’t think: Timeout → probably timing issue is enough to justify another platform. What I’d want to know is: Was the classification correct? How often is it wrong? Can it distinguish a flaky test from a real race condition? Can it identify the commit that caused the regression? Can it propose a fix that actually preserves test intent? Does it save enough investigation time to justify uploading and storing everything elsewhere? Those are much harder questions. A dashboard saying: NETWORK with a nice badge doesn’t automatically mean the debugging problem has been solved. The MCP server is probably the most interesting part This is where I finally found something I genuinely like. TestDino exposes test information to AI agents over MCP. That means your coding agent can ask things like: What failed in the latest run? Which tests are flaky? Show the historical failures for this test. What changed around this failure? without you copying logs into Claude, Cursor, or another tool manually. That’s actually interesting. I can see real value there. But again: I’m uploading my test data into a separate SaaS product so an AI can query it. That’s useful if TestDino becomes the place where my testing history lives. It’s much less compelling if I already have that information structured elsewhere. MCP doesn’t magically make mediocre data valuable. It just makes it easier for an agent to access it. Public reviews are surprisingly positive Here’s where I have to be fair. People who publicly review TestDino mostly seem to like it. Reviewers frequently praise: easy setup clean dashboards flaky-test visibility centralized reporting support So maybe I’m the grumpy one here. But there are hints of the same concerns I had. Some public feedback has called for deeper debugging, more customization, stronger documentation for advanced workflows, better linking between manual and automated tests, and more flexible repository handling. Those are not catastrophic problems. They just reinforce my overall impression: TestDino looks polished at the surface. I’m less convinced about depth. An earlier hands-on review found exactly that problem An independent review from 2025 had a similar experience. The reviewer liked the polish, ease of navigation, screenshots, videos, and support. But they also complained that dashboard metrics couldn’t be drilled into sufficiently, AI insights weren’t linked cleanly to the underlying test run or code, and PR information wasn’t well connected to test runs. To TestDino’s credit, its current documentation describes much better PR-to-run navigation, timelines, code diffs, failure clusters, logs, screenshots, and traces. So some of those problems appear to have been addressed since that review. That’s good. It also tells you the product is still evolving quickly. Whether you interpret that as: Great, they’re shipping fast. or: I’m paying to participate in the maturation process. is up to you. Then I read the Terms of Service And this may be my favorite part. TestDino’s current Terms say you may not use the platform to compile a: comparison evaluation benchmark competitive assessment without TestDino’s prior written consent. They say consent will not be unreasonably withheld for a good-faith factual comparison if the party identifies itself before requesting access. Read that again. A testing tool. A product literally designed around measuring whether software behaves correctly. Doesn’t want you evaluating the product through platform access without asking permission first. That’s certainly one approach. The terms also define competing products and people connected to them as “Restricted Parties” and say they aren’t eligible to access the platform without a written agreement. I’m not a lawyer, so I’m not going to speculate about how broadly any of that would ultimately apply. But as a buyer? I don’t love it. If I’m selecting engineering infrastructure, independent benchmarking is exactly what I want people doing. I want: test it break it compare it measure it publish the results If a vendor wants people to request permission before using the product for an evaluation or benchmark, that’s a yellow flag for me all by itself. Ironically, this makes an “honest hands-on review” harder This is also why I wouldn’t pretend I spent a week hammering the private product and publish invented observations. TestDino’s publicly available showcase gives a detailed look at streaming results, run summaries, PR coverage, flaky detection, error grouping, test management, test explorer, tags, and imports/exports. Its documentation is extensive. There are third-party tutorials and reviews. But there’s a difference between: I inspected the product’s public surfaces, documentation, pricing, demos, independent reviews, and technical architecture. and: I personally ran 10,000 tests through it. I’m not going to pretend those are the same thing. Any review that does should immediately lose credibility. So who actually needs TestDino? I can think of one very specific team. You have: a large Playwright-only test suite + lots of CI runs + multiple engineers investigating failures + persistent flaky-test problems + poor historical reporting + a desire to expose test history to AI agents Then yes. I can understand TestDino. You’re buying organization. You’re buying history. You’re buying triage. You’re buying convenience. Fine. But if you’re a small or medium Playwright team already keeping useful CI artifacts? I’d have a hard time recommending it. You already have a very capable testing framework. You already have traces. You already have screenshots. You already have videos. You already have CI history. Now you’re adding: another account another reporter another dashboard another API key another SaaS bill another usage limit another retention policy another copy of your test data to make those things look nicer. My biggest problem with TestDino It’s not that TestDino appears completely useless. That would actually make the review easier. There are useful ideas here. The MCP integration is interesting. Cross-run flaky-test analytics are valuable. Centralized evidence can save time. The product looks polished. The problem is that the value proposition feels incredibly thin. It feels like someone looked at Playwright’s reporting output and said: What if this was a startup? And now we have pricing tiers, AI classification, usage quotas, retention windows, team-member limits, enterprise redaction, and a Terms of Service section about benchmarking. I wanted to find the killer feature. The thing where I went: Oh. That’s why I’d install this. I didn’t find it. My verdict For a large Playwright-only organization drowning in CI results: Maybe. For everyone else: I’d skip it. My rough score: UI and polish: 8/10 Setup: 8/10 Reporting: 7/10 AI differentiation: 4/10 Framework coverage: 2/10 Pricing model: 5/10 Artifact retention: 4/10 Need to exist: 3/10 The harshest thing I can say about TestDino isn’t that it looks badly built. It doesn’t. It’s that after going through everything it offers, I kept coming back to the same question: Why isn’t this just part of my existing Playwright and CI workflow? And when a developer tool makes me ask that question this many times, adding another SaaS subscription is probably not the answer.