Funny

8796 readers

908 users here now

General rules:

Be kind.
All posts must make an attempt to be funny.
Obey the general sh.itjust.works instance rules.
No politics or political figures. There are plenty of other politics communities to choose from.
Don't post anything grotesque or potentially illegal. Examples include pornography, gore, animal cruelty, inappropriate jokes involving kids, etc.

Exceptions may be made at the discretion of the mods.

founded 2 years ago

MODERATORS

[email protected]

528

So, how's the job search going? (lemmy.world)

submitted 4 days ago by [email protected] to c/[email protected]

53 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 15 points 2 days ago (2 children)

Yes, I actually do. It's a behavioral driven testing framework. I love the idea of it but hate working with it because everyone uses it improperly.

What the steps in a scenario should look like:

Given a user has a bank accout
And the account has a balance of $10
When the user attempts to withdraw $20
Then the transaction should fail

How everyone I've worked with uses it:

Given the system is setup
Given the user TESTUSER1
Given load TESTDATA1
When the user sends a request
    ACTION | AMOUNT
    WITHDRAW | 20.0
The the response should be 400
    STATUS | MESSAGE
    Failure | Not enough funds

[–] [email protected] 1 points 2 days ago (1 children)

Your (verbose and harder to understand) example reminds me of https://jbehave.org/reference/stable/story-syntax.html

[–] [email protected] 2 points 2 days ago* (last edited 2 days ago)

On that same page is Gherkin which is what both examples are. (I may have gotten the syntax slightly wrong.) Cucumber uses Gherkin. I forget which is which exactly. Maybe Cucumber is the code that reads Gherkin and executes step definitions.

For whatever reason, people try to make a small number of extremely flexible step definitions which leads to scenarios that don't actually read as business requirements. (Which is the entire point in the first place.)

Given a user has a bank accout
And the account has a balance of $10
When the user attempts to withdraw $20
Then the transaction should fail
And the account should have a balance of $20

Given a user has a bank accout
And the account has a balance of $10
When the user attempts to withdraw $5
Then the transaction should succeed
And the account should have a balance of $5

Given a user has a bank accout
And the account has a balance of $10
When the user attempts to deposit $5
Then the transaction should succeed
And the account should have a balance of $15

Doing something like this is more elegant. The steps definitions would be

a user has a bank account performs setup tasks to load a test user into the system.
the account has a balance of X would either load the account with that much or just make an initial deposit.
the user attempts to withdraw X/the user attempts to deposit X would both perform those actions. If it's a web API they'd be HTTP requests.
then the transaction should X would check HTTP status code. You could make this two separate step definitions or have logic based on the word. Two separate step definitions is probably better.
the account should have a balance of X checks the balance in the account.

[–] [email protected] 1 points 2 days ago (2 children)

Maybe that's why I hate using it, or maybe I just hate testing frameworks that do more than run tests. Here's what I want from a testing framework:

run tests in classes or modules
report output with some configurable name (javadoc comment, Python Docstring, etc)
some way to parameterize tests with different inputs
organized output that shows what failed
if it's an integration test, keep track of failures over time

Our QA uses cucumber and it works for them, so I only whine when I need to deal with it.

[–] [email protected] 1 points 2 days ago (1 children)

Cucumber can do all of that. Except for failures over time but I've never used a framework for that. CI tools typically track that.

[–] [email protected] 1 points 2 days ago (1 children)

Sure, and my issue isn't that it's capable of my requirements, but simple to use and meets those requirements. Simplicity is a quality all its own.

JUnit provides sufficient output to meet my requirements, so why overcomplicate it w/ Cucumber? AFAIK, cucumber tests in Java output JUinit as well (at least ours does), so what value exactly is Cucumber providing?

[–] [email protected] 1 points 2 days ago (1 children)

If you don't want the scenarios that are readable in a business requirement format then it's not worth the trouble. I haven't ever seen anyone use it in that way.

[–] [email protected] 1 points 2 days ago

Yeah, I don't get it. The only people looking at the results are devs, so we really don't need the business requirement text.

I could maybe understand if product owners were creating the requirements in the feature files and devs were implementing them in the code, but that never happens. So the whole cucumber library thing we have going on just feels annoying.

It's not the only thing we over-engineer. We have dependency injection when we only ever have one dependency (except in tests, in which case we could just use mocking). We take microservices to the extreme (we just broke out a couple hundred lines of code from a few thosand lines of code service). Our test code is unnecessarily reusable (test code is one place where DRY doesn't really need to apply). And so on. This one in particular is especially annoying because it makes it harder to find the code without providing much benefit.

Our QAs seem to like it, so whatever. I'll still complain though.

[–] [email protected] 1 points 2 days ago (1 children)

I think there's one question you should answer in order to fully describe your "testing framework": is it being used for "end to end tests" or "integration tests" or "unit tests"? https://k-hartanto.medium.com/testing-pyramid-and-testing-ice-cream-cone-what-is-the-difference-6ddde3876c20

For unit tests, something like https://en.wikipedia.org/wiki/JUnit is useful. For testing a program after it's deployed, something like https://jbehave.org/reference/stable/story-syntax.html is useful. You get different information about your program from each type of testing, and one type can detect issues even if the other didn't, so doing both is useful.

[–] [email protected] 2 points 2 days ago

These are for a mix of end to end and integration tests.

I mostly do unit tests as a dev, so our tests are simple enough that they don't benefit from more structure than being grouped by suite. E.g.:

AuthService
- valid user creds can login
- invalid user creds cannot login
- non-existent user gets same error as wrong creds
UserSettingsService
- can change language
- cannot set empty password

These don't have long flows, so there's no benefit to documenting steps (they usually have one step).

My complaint about cucumber/gherkin isn't with documenting steps, it's with managing them in separate files. We have a Service.feature file that documents the scenario and the ServiceTest.java that documents the steps. I don't see the point in having those be separate files, especially since the only people defining inputs and scenarios are the devs (dedicated QA in our case). We occasionally have our BE devs help write a few tests, and every time it's a struggle for them to figure out where everything is. It just feels over-engineered.

In unit tests, we parameterize our tests just like with cucumber, we just do so in the code. E.g. in Python:

@parameterized.expand([(1, 2), (2, 4)])
def test_duplicate(num, exp):
    res = dup(num)
    assert res == exp

I would much prefer something like that in our end to end and integration tests.