Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Region Capture #710

Closed
1 task done
eladalon1983 opened this issue Jan 20, 2022 · 3 comments
Closed
1 task done

Region Capture #710

eladalon1983 opened this issue Jan 20, 2022 · 3 comments
Assignees
Labels

Comments

@eladalon1983
Copy link

eladalon1983 commented Jan 20, 2022

Braw mornin' TAG!

I'm requesting a TAG review of Region Capture.

An API for cropping a video track derived from display-capture of the current browsing context.

Further details:

  • I have reviewed the TAG's Web Platform Design Principles
  • Relevant time constraints or deadlines: N/A
  • The group where the work on this specification is currently being done: WebRTC WG
  • Major unresolved issues with or opposition to this specification: N/A
  • This work is being funded by: Google

We'd prefer the TAG provide feedback as:
🐛 open issues in our GitHub repo for each point of feedback

@cynthia cynthia self-assigned this Feb 1, 2022
@torgo torgo added Venue: WebRTC WebRTC and media capture and removed Progress: untriaged labels Feb 1, 2022
@torgo torgo self-assigned this Feb 1, 2022
@torgo torgo added this to the 2022-02-07 milestone Feb 1, 2022
@cynthia
Copy link
Member

cynthia commented Feb 8, 2022

First of all, this looks pretty useful.

API surface-wise, what happens when one tries to create a crop target on a element that is not attached to a document, hence not visible? (I assume it can be rendered and therefore can be captured, but seems like a bug or an interesting side-effect feature?)

I see benefits from having the element being the region boundary, but also a bit unconventional. Initially we thought this was replicating what native does and provides a bounding box to be streamed, but that does not seem to be the case here. This feels like a user expectation that should be fulfilled.

We weren't able to figure out the user flow of how the preview is presented the user in the example code. Is it triggered when the crop target promise is being returned? (Presumable rejecting if the user dismisses the preview as "please don't"?)

@eladalon1983
Copy link
Author

eladalon1983 commented Feb 8, 2022

First of all, this looks pretty useful.

Thanks. :-)

API surface-wise, what happens when one tries to create a crop target on a element that is not attached to a document, hence not visible? (I assume it can be rendered and therefore can be captured, but seems like a bug or an interesting side-effect feature?)

The thing that happens if one creates a CropTarget for an element that is attached, but then detaches the element. Namely, the track is muted¹.

Initially we thought this was replicating what native does and provides a bounding box to be streamed, but that does not seem to be the case here.

I think you initial interpretation was in fact correct. The element defines a bounding box to be streamed.

This feels like a user expectation that should be fulfilled.

The user is not in control of the feature; the application is. The application could be saving the cropped file to disk, storing it remotely, doing OCR inside of it... Anything, really. Whether it's user-facing or not is up to the application, and not up to the API.

We weren't able to figure out the user flow of how the preview is presented the user in the example code.

Could you please clarify the question? What preview are you asking about? (Note clarification above.)

--
[1] The muting part is currently under discussion. It's unclear at the moment if we'll set MediaStreamTrack.muted, but regardless, no frames will be delivered, so "effectively muted."

@torgo torgo modified the milestones: 2022-02-07, 2022-02-14-week, 2022-02-21-week Feb 10, 2022
@torgo torgo added the Progress: propose closing we think it should be closed but are waiting on some feedback or consensus label Feb 22, 2022
@torgo torgo modified the milestones: 2022-02-21-week, 2022-02-28-week Feb 24, 2022
@cynthia
Copy link
Member

cynthia commented Mar 1, 2022

Thank you for the clarification!

I think you initial interpretation was in fact correct. The element defines a bounding box to be streamed.

I think there is a minor misunderstanding here, we understood this as a bounding box (draw a rectangle) in any part of the desktop - which is not what this proposal is about.

Given that the naming/expectations might introduce confusion - may we suggest renaming this to something that suggests a specific scope? (e.g. "Tab" Region Capture)

Additionally, we'd like the group to consider the unfulfilled user need of "stream arbitrary region from any part of the screen" in WebRTC at some point.

Thank you for bringing this to our attention, and we are happy to see this proposal move forward.

@cynthia cynthia closed this as completed Mar 1, 2022
@cynthia cynthia added Resolution: satisfied The TAG is satisfied with this design and removed Progress: propose closing we think it should be closed but are waiting on some feedback or consensus labels Mar 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants