Write a suite of user experience end-to-end integration tests for WikiLambda in selenium/etc. sufficient for us to launch with
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	DVrandecic
	Mar 8 2021, 5:59 PM

Description

We should have a framework for integration tests, that ensure the different components work together end-to-end. That includes the front end, the wiki, and the evaluator (orchestrator and executor).

1. Evaluate a function
2. Create a function definition
3. Edit a function definition T321934
4. Create testers T318938
5. Create implementations T318939
- 5a. Create implementations via code T321937
- 5b. Create implementation via composition T321938
6. Connect an implementation or test to a function T321939
~~7. Create and edit types T321940~~

Details

Subject	Repo	Branch	Lines /-
Write a suite of user experience end-to-end integration tests	mediawiki/extensions/WikiLambda	master	115 -1
selenium: Install missing npm dependencies	mediawiki/extensions/WikiLambda	master	406 -54
Add initial selenium tests	mediawiki/extensions/WikiLambda	master	4 K -3 K

Customize query in gerrit

Related Objects
Search...

Status	Subtype	Assigned	Task
Open	BUG REPORT	None	T344206 Special:MathStatus exception error on Wikifunctions
Resolved		Jdforrester-WMF	T342865 Post-creation work for wikifunctionswiki
Resolved		BTullis	T289316 Prepare and check storage layer for Wikifunctions.org (new public content wiki)
Resolved		Jdforrester-WMF	T278635 Create Wikifunctions.org
Resolved		Jdforrester-WMF	T299176 Phase Theta – Root task
Resolved		SDunlap	T294388 Run some full end-to-end browser integration tests on Beta Cluster Wikifunctions now it exists
Resolved		SDunlap	T276847 Write a suite of user experience end-to-end integration tests for WikiLambda in selenium/etc. sufficient for us to launch with
Resolved		Teleosteen	T321932 E2E test for CUJ 1: Evaluate a function
Resolved		Teleosteen	T321933 E2E test for CUJ 2: Create a function definition
Resolved		DVrandecic	T318930 CUJ 2: Create a function definition
Resolved		DVrandecic	T323804 Design: create a function definition
Resolved		Ddwaal	T321399 Don't show / disable the "show more languages" / "Show name in other languages" buttons when there aren't any to show
Resolved		Ddwaal	T321396 Copy: Evaluate copy on function viewer details tab
Resolved		Nik.xyz.in	T321934 E2E test for CUJ 3: Edit a function definition
Resolved		DVrandecic	T318933 CUJ 3: Edit a function definition
Resolved		DVrandecic	T318314 Function editor: Remove an item from the list of arguments
Resolved		• AdesojiThisDot	T318315 Function editor: Implement functionality of "remove input field" using the Mobile view button.
Resolved		Ddwaal	T318317 Function editor: Implement "remove input field" button for Desktop view
Resolved		AAlhazwani-WMF	T312121 The dropdown menu of the "output type" from the function definition page is hidden under another component
Resolved		None	T323805 Design: edit a function definition
Resolved		• aishwaryavardhana	T285418 A way to easily and intuitively create and edit functions
Resolved		• aishwaryavardhana	T285419 Initial user flow diagrams for function editor
Invalid		• aishwaryavardhana	T285421 Generative design research: Testing of new hypotheses and mental models
Declined		• SWoodruff-WMF	T285422 Function editor: User education, interface copy, notifications and alerts
Declined		• SWoodruff-WMF	T285684 As someone who is new to creating functions, I would like to be able to start with an easy task
Resolved		• aishwaryavardhana	T296693 Desktop designs for function editor v1
Resolved		• AdesojiThisDot	T297086 Function Editor - Sidebar - Language selector
Declined		• AdesojiThisDot	T298203 Provide an easy way on the front-end for Wikifunctions to know what languages are supported (and what their ZIDs are)
Resolved		SimoneThisDot	T297095 Function Editor - Function Definition - New component
Resolved		• JKieserman	T297098 Function Editor - Function definition - Heading
Resolved		• AdesojiThisDot	T297099 Function Editor - Enhance the "Object selector" to allow user to type within its first entry instead than the input itself to align with the designs
Resolved		SimoneThisDot	T297100 Function Editor - Create the function definition main content area
Resolved		• AdesojiThisDot	T297101 Function Editor - Allow user to add labels in another language in function definition
Resolved		• JKieserman	T297104 Function Editor - Provide the ability for users to publish, write implementation and cancel function
Resolved		• JKieserman	T297271 Function Editor - Create an Chip component
Resolved		• AdesojiThisDot	T297280 Function Editor - sidebar - Update diagram
Resolved		gengh	T297869 Function Editor: Allow user to REMOVE an Input from a function definition
Resolved		SimoneThisDot	T298464 Remove old function editor and its files
Resolved		• aishwaryavardhana	T296698 Provide a UX design for function viewing
Resolved		SimoneThisDot	T301518 Explore replacing the legacy function editor with fallback to the generic editor via a button, so that functions can be connected to implementations and testers and we can scrap the legacy function editor
Resolved		• aishwaryavardhana	T296694 Mobile designs for function editor v1
Resolved		• aishwaryavardhana	T296696 Provide a compelling, scalable, validated UX design for function creation and editing
Resolved		• aishwaryavardhana	T297458 User flow for when a user edits definition
Resolved		• aishwaryavardhana	T297462 User flow when a user only edits labels
Resolved		• aishwaryavardhana	T297463 User flow for how a user would complete multilingual edits
Resolved		None	T297464 Decide on the UX design for the edit summary
Resolved		• aishwaryavardhana	T297465 Design notes for if an input type is changed
Resolved		• aishwaryavardhana	T297466 Design notes for if an output type is changed
Resolved		• aishwaryavardhana	T297467 Decide on the UX design for attaching and detatching (but not editing) implementations and testers
Resolved		• aishwaryavardhana	T297668 Revisit user flows for function editor
Resolved		• SWoodruff-WMF	T318316 Function editor: Design function argument (input) delete button/icon for Desktop view.
Resolved		AAlhazwani-WMF	T320410 Design: Function editor, redefine separation between fields and blocks.
Resolved		AAlhazwani-WMF	T270304 Design a "Publish" component
Resolved		SimoneThisDot	T268296 Display to user why an object is not stored
Duplicate		None	T274424 Show errors if creation of ZObject fails
Resolved		DVrandecic	T316686 Copy: change text for confirmation dialog when creating a new function on mobile
Resolved		DVrandecic	T303336 Users can write an edit summary when publishing which can be seen by other users in histories
Resolved		DVrandecic	T299004 Have the publish function enable/disable itself only if there have been changes
Resolved		gengh	T326656 FunctionEditor: Implement separation between fields and blocks changes
Resolved		Nik.xyz.in	T321935 E2E test for CUJ 4: Create tester for function
Resolved		DVrandecic	T318938 CUJ 4: Create tests
Invalid		None	T323806 Design: create tests
Resolved		Nik.xyz.in	T321937 E2E test for CUJ 5: Create implementations with code
Resolved		Nik.xyz.in	T321939 E2E test for CUJ 6: Connect an implementation or test to a function
Resolved		DVrandecic	T318936 CUJ 6: Connect an implementation or test to a function
Resolved		DVrandecic	T323808 Design: connect an implementation or test to a function
Resolved		Ddwaal	T320278 Copy: Language for "connecting" an implementation or tester to a ZFunction
Resolved		Ddwaal	T318426 Tests: Implement FE integration test for connecting implementations and testers to functions
Resolved		SDunlap	T321942 make E2E test suit runnable via "helm test"
Resolved		Nik.xyz.in	T321938 E2E test for CUJ 5: Create implementations with composition
Resolved		Nik.xyz.in	T318939 CUJ 5: Add implementations via code or composition
Invalid		None	T323807 Design: add implementations via code or composition

Event Timeline

DVrandecic created this task.Mar 8 2021, 5:59 PM

Jdforrester-WMF renamed this task from Integration tests to Write a suite of user experience end-to-end integration tests for WikiLambda in selenium/etc..Mar 8 2021, 6:55 PM

DVrandecic moved this task from To Triage to Phase ζ on the Abstract Wikipedia team board.Mar 10 2021, 5:21 PM

DVrandecic edited projects, added Abstract Wikipedia team (Phase ζ); removed Abstract Wikipedia team.

Change 697056 had a related patch set uploaded (by Simone Cuomo; author: Simone Cuomo):

[mediawiki/extensions/WikiLambda@master] Add selenium to Repository

https://gerrit.wikimedia.org/r/697056

gerritbot added a project: Patch-For-Review.May 28 2021, 1:23 PM

zeljkofilipin added a project: User-zeljkofilipin.May 28 2021, 1:58 PM

zeljkofilipin moved this task from Backlog 🪒 to Watching 📺 on the User-zeljkofilipin board.

I've left a couple of comments on the patch. Let me know if you need any help.

Sahilgrewalhere subscribed.May 31 2021, 1:37 PM

Change 697056 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] Add initial selenium tests

https://gerrit.wikimedia.org/r/697056

Maintenance_bot removed a project: Patch-For-Review.Jun 1 2021, 9:10 PM

DVrandecic moved this task from Phase ζ to To triage: Un-phased work on the Abstract Wikipedia team board.Jun 2 2021, 4:42 PM

DVrandecic edited projects, added Abstract Wikipedia team; removed Abstract Wikipedia team (Phase ζ).

To run a full end-to-end test in CI, we'd need T280070: Provide a CI mechanism by which we can test end-to-end integration between MW and back-end services, e.g. via docker-compose to be resolved first.

DVrandecic assigned this task to SimoneThisDot.Jun 9 2021, 4:43 PM

DVrandecic triaged this task as High priority.

Change 699736 had a related patch set uploaded (by Simone Cuomo; author: Simone Cuomo):

[mediawiki/extensions/WikiLambda@master] Write a suite of user experience end-to-end integration tests

https://gerrit.wikimedia.org/r/699736

gerritbot added a project: Patch-For-Review.Jun 14 2021, 11:57 AM

Change 702120 had a related patch set uploaded (by Zfilipin; author: Zfilipin):

[mediawiki/extensions/WikiLambda@master] selenium: Install missing npm dependencies

https://gerrit.wikimedia.org/r/702120

Change 702120 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] selenium: Install missing npm dependencies

https://gerrit.wikimedia.org/r/702120

zeljkofilipin moved this task from Watching 📺 to Deep work 🌊 on the User-zeljkofilipin board.Jul 2 2021, 1:29 PM

Lindsaykwardell mentioned this in T286228: Expand E2E test coverage.Aug 4 2021, 5:33 PM

Jdforrester-WMF merged a task: T286228: Expand E2E test coverage.Aug 4 2021, 5:53 PM

Jdforrester-WMF added subscribers: SimoneThisDot, Lindsaykwardell.

In T276847#7129612, @Jdforrester-WMF wrote:

To run a full end-to-end test in CI, we'd need T280070: Provide a CI mechanism by which we can test end-to-end integration between MW and back-end services, e.g. via docker-compose to be resolved first.

Is this task still blocked?

zeljkofilipin moved this task from Deep work 🌊 to Watching 📺 on the User-zeljkofilipin board.Aug 24 2021, 2:22 PM

To the best of my knowledge, yes.

Jdforrester-WMF removed SimoneThisDot as the assignee of this task.Sep 14 2021, 8:17 PM

Jdforrester-WMF edited projects, added Abstract Wikipedia team (Phase κ – Clean-up); removed Abstract Wikipedia team.

zeljkofilipin added a project: Browser-Tests.Oct 7 2021, 11:12 AM

SDunlap claimed this task.Oct 17 2022, 3:17 PM

Jdforrester-WMF moved this task from Incoming to Blocking launch on the Abstract Wikipedia team (Phase κ – Clean-up) board.Oct 24 2022, 8:24 PM

Jdforrester-WMF added a parent task: T294388: Run some full end-to-end browser integration tests on Beta Cluster Wikifunctions now it exists.Oct 25 2022, 1:43 PM

SDunlap mentioned this in T321405: Proposal: end-to-end tests with Gherkin syntax and automated browsers.Oct 25 2022, 9:37 PM

SDunlap merged a task: T321405: Proposal: end-to-end tests with Gherkin syntax and automated browsers.Oct 28 2022, 6:38 PM

SDunlap added subscribers: Teleosteen, SDunlap.

Jdforrester-WMF closed subtask T321932: E2E test for CUJ 1: Evaluate a function as Resolved.Nov 29 2022, 7:52 PM

Change 699736 abandoned by Simone Cuomo:

[mediawiki/extensions/WikiLambda@master] Write a suite of user experience end-to-end integration tests

Reason:

This is too old to be relevant and also would be harder to update with current code than start from scratch

https://gerrit.wikimedia.org/r/699736

Jdforrester-WMF removed a subtask: T321938: E2E test for CUJ 5: Create implementations with composition.Jan 5 2023, 4:22 PM

SDunlap closed subtask T321942: make E2E test suit runnable via "helm test" as Resolved.May 5 2023, 5:58 PM

Maintenance_bot removed a project: Patch-For-Review.May 5 2023, 6:10 PM

Jdforrester-WMF closed subtask T321933: E2E test for CUJ 2: Create a function definition as Resolved.Jun 1 2023, 6:01 PM

Jdforrester-WMF closed subtask T321935: E2E test for CUJ 4: Create tester for function as Resolved.

Jdforrester-WMF closed subtask T321937: E2E test for CUJ 5: Create implementations with code as Resolved.Jun 6 2023, 7:30 PM

Jdforrester-WMF moved this task from Phase κ – Clean-up to Phase λ – Launch on the Abstract Wikipedia team board.Jun 16 2023, 8:38 AM

Jdforrester-WMF edited projects, added Abstract Wikipedia team (Phase λ – Launch); removed Abstract Wikipedia team (Phase κ – Clean-up).

Jdforrester-WMF moved this task from Incoming to In progress on the Abstract Wikipedia team (Phase λ – Launch) board.Jun 16 2023, 8:54 AM

Jdforrester-WMF removed a subtask: T330702: Extend the CUJ1 browser test asserting that a function can be called and returns a result to ensure we can adjust input values.Jun 20 2023, 12:37 PM

Jdforrester-WMF removed a subtask: T321940: E2E test for CUJ 7: Create and edit types.

Jdforrester-WMF renamed this task from Write a suite of user experience end-to-end integration tests for WikiLambda in selenium/etc. to Write a suite of user experience end-to-end integration tests for WikiLambda in selenium/etc. sufficient for us to launch with.Jun 20 2023, 12:45 PM

Jdforrester-WMF updated the task description. (Show Details)

Jdforrester-WMF added a parent task: T299176: Phase Theta – Root task.Jun 20 2023, 1:00 PM

Jdforrester-WMF added a subtask: T321938: E2E test for CUJ 5: Create implementations with composition.Jun 20 2023, 3:49 PM

Jdforrester-WMF updated the task description. (Show Details)Jun 20 2023, 7:23 PM

Jdforrester-WMF closed subtask T321938: E2E test for CUJ 5: Create implementations with composition as Resolved.Jul 5 2023, 1:15 PM

Jdforrester-WMF closed subtask T321939: E2E test for CUJ 6: Connect an implementation or test to a function as Resolved.

Jdforrester-WMF updated the task description. (Show Details)

DVrandecic closed subtask T321934: E2E test for CUJ 3: Edit a function definition as Resolved.Jul 6 2023, 12:41 AM

DVrandecic updated the task description. (Show Details)

DVrandecic updated the task description. (Show Details)Jul 6 2023, 12:50 AM

DVrandecic closed this task as Resolved.Jul 6 2023, 12:52 AM

Jdforrester-WMF added a project: Epic.Jul 10 2023, 1:36 PM