Writer/ToDo/TrackChanges

Writer Track Changes Enhancement

On Path to Vision

Author: chengjh/Oliver/zhengfan(@apache.org) Date:June 13, 2012 Status:Draft/Reviewing/Approved

Overview

This document will focus on studying Track Changes of Apache OpenOffice.org Writer, and try to explore better design to improve the function area for expected interoperability with MS Word. Additionally, Track Changes of Apache OpenOffice.org Spreadsheet is beyond the scope and will not be impacted.

Requirement

• From legal community

The track changes area is demanded to get 100% interoperability,high fidelity with MS Word. Otherwise, they will never use Symphony/Apache OpenOffice.org. And that means contents cannot move to different pages comparing with MS Word because there are references within the legal contracts that must be accurate enough.

The related function area is “compare/merge documents”. End users want to open two documents at the same time and see the differences between them. It is like a version thing. They have their old document, and saved it as old, then made some changes, saved it as new. At last, they want to open both in the same screen and are able to see the differences. Spreadsheet is also asked to provide the capability.

• From Certain Application to Handling ODT Document

If the data of change records can be separated. That will benefit such kind of applications to handle the change record data from document. For example, the record data can be seperated converiently and also can be applied to the original document to get the final results.

• From Customers/Known Defects

S/N	Brief Description	Severity	Priority
1#	[Feature Gap]Unable to Print Comments generate in Edit Revisions	4-Hindrance	P3
2#	The contents are distorted in the sample doc file	2-Severely Restricted	P2
3#	[L3][Legal][Feature][Reviewing]Formating of Track Changes is not supported in Sym1.3 and 2.0	3-Restricted	P1
4#	Deletion of field lost, in both Windows & Linux	3-Restricted	P3
5#	[FFC]the font of the texts that revisioned in MS Word can not marked as the Format action in SD.	3-Restricted	P3
6#	The recorded format can not be reject	3-Restricted	P2
7#	The record formatting change lost when merge document	3-Restricted	P2
8#	The changes of the text in Header are accept automatically after create Footer in the doc.	3-Restricted	P2
9#	Can not merge document if the changes only appear in frame, header and footnote	3-Restricted	P3
10#	All comments lost and some revision text mess up with the sample file	3-Restricted	P2
11#	TOC is not correct for revision record with doc file	3-Restricted	P1

2#:

11#:

Note:

In Apache OpenOffice.org Writer,the TOC will be updated after being loaded...the page numbers will not be duplicated,otherwise,the result will be same with Symphony..but the track changes of TOC are still lost.

Problem Description

• Interoperability ProblemsWhen loading a MS Word document containing track changes,some records may be lost,position of recorded flying objects will be lost,contents layout may be disordered.
• Core Functionality ProblemsCurrently,Symphony/AOO only provides limitation track changes functions,only supports to record basic contents and attributes. Comparing with MS Word, obvious function GAPs are existing. Moreover, the design of the function area has problem too.
• Stability ProblemsBecause of the function and design limitation, once flying objects exist within recorded contents, the operations of these redlines will lead to stability problems.
• Other Related ProblemsComments management problem and print problem,presentation problem of deletion records...

Stability Problem Example:

Description:

1.New a SW document,insert a table or frame into the doc.

2.Click "Edit->Changes->Record" to check it.

3.Press "Ctrl A" to select all, then press "Delete" key to delete the selected content.(The frame or table will marked deleted tag).

4.Insert a graphic into one cell or frame.

5.Save the doc or then undo,crash

Analysis and Root Cause:

This problem is caused by the conflict of design between Change Record and flying object related undo action;

In Writer of OpenOffice, a flying object related undo action is implemented based on a pointer of SwFlyFrmFmt, which has a reference to the actual flying object. In order to ensure the validation of the pointer, the destruction of the pointer is managed by the undo action. For example, if a flying object is removed by end user, the corresponding SwFlyFrmFmt is not be destructed accordingly, but removed from the flying frame format list of the document (SwDoc::pSpzFrmFmtTbl), and said SwFlyFrmFmt is maintained by the undo action only;

But on the other hand, the change record (Reviewing) will delete SwFlyFrmFmt directly in corresponding range of a "Delete" type record without doing undo action correspondingly while changing the mode option of change record from "Show" to "Hide"(That is reasonable, for changing the Change Record options from "Show" to "Hide" make no sense for users as a modification). Said directly deleting of SwFlyFrmFmt will makes the SwFlyFrmFmt pointer referred in flying object related undo actions (if there are some) to be a dangling pointer, and the crash will happens sooner or later;

In general, root cause of said problem could be summarized as: an object is managed by 2 conflict patterns.

Comparison and GAP

GAP from UI Perspective

Although MS Office 2007/2010 uses ribbon, which is different against 2003, the interfaces are almost the same.

MS Office		Apache OpenOffice.org Writer
Track Changes		Track Changes
	* Track Changes (on/off)		* Edit->Changes->Record
	* Change Tracking Options (Detailed Settings)		* Tools->Options...->AOO Writer->Changes
	* Change User Name		* Tools->Options...->AOO->User Data (But the new user name can only be applied to Note, not changes' comments.)
Balloon
	* Show Revisions in Balloons
	* Show All Revisions Inline		* All Changes are Inline, when the changes are not hidden.
	* Show Only Comments and Formattings in Balloons
Display Options
	Final Showing Markup; Final; Original Showing Markup; Original.		By “Edit->Changes->Show”, Final Showing Markup; Final.
	Show what type of Markups? Show which reviewer's markups?		Show ALL changes (Deletions, Insertion, Formatting) of all reviewers.
	Reviewing Pane
Accept/Reject
	Buttons on Ribbon Toolbar		Edit->Changes->Accept or Reject...
Comment
	Comment are shown by side-balloons		There are 2 kinds of comment in Sym. Edit->Changes->Comment Can only be viewed by “Edit->Changes->Accept or Reject...” Insert->Comment
Compare
	Compare		Edit->Compare Document...
	Combine		Edit->Changes->Merge Document... Showing reviewer's changes of both documents.
	Show Source Documents (Show the 2 documents in windows)

Remarks:

All changes, comments can be navigated through by buttons.

Some hot functions should have an entry from toolbar/sidebar, i.e., Change Record On/Off, Comment, Accept/Reject, etc.

GAP from Contents

Overview

• Common Contents and Attributes Supported:[Create/Modify/Delete]

1. 1. Characters
   2. Character attributes
   3. Paragraph [Except Paragraph Formats]
   4. Table [Except Table Merge/Cell Merge/Cell Split/Insert Column/Delete Column/Insert Row/Delete Row/Adjust Column Width/Adjust Row Height...]
   5. Notes/Comments [Except the contents add/delete,formats changes of the contents within notes/comments]

• Contents un-supported:[Create/Modify/Delete]

1. 1. Textbox
   2. Drawings
   3. Graphic
   4. Frame [Except the contents add/delete and character formats modification]
   5. OLE
   6. Contents within Flying Objects/1),2),3),5)...
   7. Styles [Except character styles]
   8. Bullets and Numbering
   9. Page Attributes
   10. Document Attributes
   11. Paragraph Attributes

• Behaviors Differences

1. 1. All deleted contents are marked with “Strikethrough” lines. If the “Edit->Changes->Record” is checked, the deleted contents will still there. And MS Word has move all deleted contents into comments display area by default. So,that will lead to the different layout when loading a MS Word document with tracking changes.
   2. MS Word supports to track changes on the contents un-supported by Symphony/AOO above,especially, to the “Flying” contents. Once the “Flying” contents are deleted,they will have no places in a MS Word document,but Symphony/AOO has to provide positions and display them,which also leads to layout differences.
   3. Different Strategy to make change record activity...

Details

S/N	Item Description	MS Word 2003	AOO	Comments
1.1#	Create Text /*including copy and paste*/	Y	Y
1.2#	Create Table /*including copy and paste*/	Y	Y
1.3#	Create Graphic/Frame/Drawing/Form/...Flying..	Y	N	If the created drawing is out of the painting canvas,it will not be recorded...not sure whether it is a word's bug..
1.4#	Create fields/.../content that is within SwPaM data structure	Y	Y
1.5#	Create Header/Footer	N	N	In MS Word,Header/Footer can only be created through inputting content into Header/Footer,otherwise,Header/Footer will not be created..
1.6#	Create Footnote and Endnote	Y	Y
1.7#	Create Break(Line Break/Column Break/Page Break)	Y	Y
1.8#	Create Comments	Y	Y
2.1#	Delete Text /*including copy and paste*/	Y	Y
2.2#	Delete Table /*including copy and paste*/	Y	Y
2.3#	Delete Graphic/Frame/Drawing/Form/...Flying..	Y	N	If the created drawing is out of the painting canvas,it will not be recorded...not sure whether it is a word's bug..
2.4#	Delete fields/.../content that is within SwPaM data structure	Y	Y
2.5#	Delete Header/Footer	N	N	In MS Word,Header/Footer can only be created through inputting content into Header/Footer,otherwise,Header/Footer will not be created..
2.6#	Delete Footnote and Endnote	Y	Y
2.7#	Delete Break(Line Break/Column Break/Page Break)	Y	Y
2.8#	Delete Comments	Y	Y
3.1#	Text Properties Change	Y	Y
3.2#	Format Text through Text Style	Y	Y
3.3#	Paragraph Properties Change	Y	N
3.4#	Bullet/Numbering Paragraph	Y	N
3.5#	Format Paragraph through Paragraph Style	Y	N
3.6#	Flying Object(Graphic/Drawing/Frame/Form..) Properties Change	N	N
3.7#	Adjust Table Column/Row	Y	N
3.8#	Page Properties Change	Y	N
4.1#	Deleted Content Moved to the Side	Y	N
4.2#	Consistent Comments Feature	Y	N	Comments within Track Changes in Symphony/AOO is different with the Comments Feature
5.1#	How are you How are you How are you MS: Record three actions Symphony/AOO: one action if within given time	Y	Y	Different Strategy to record the changes

Goals

1. Ensure the high fidelity Interoperability with MS Word,MS Word 2003 is the first and 2010 is the second.
2. Enhance the Function Area to eliminate the GAPs
3. Ensure the Stability
4. Ensure the Usability

Proposal of Achieving the Goals

We will finish the initial investigation,solution proposals and prototype of conception...and then propose the project to Apache OpenOffice.org community to recruit volunteers. The final design and implementation will be released with AOO.

Currently, we are on the investigation of solution proposals and prototype of conception.

Current Design

Current Class Diagram

This is the Redline class diagram. From the inherited relationship of Redline, we can see Redline is a child class of SwPaM, which determines its capabilities directly, along with the capabilities of Symphony/AOO's tracking changes.

Procedure of Insert Action

Procedure of Delete Action

Similar with 5.2.

Procedure of Show Action

Call “MoveFromSection” to move the contents of delete records from the given section to the document body.

This procedure will be invoked when checking “Edit->Changes->Show”, and also when saving document.

Procedure of Hide Action

Call “MoveToSection” to move the contents of delete records from the document body to the given section .

This procedure will be invoked when un-checking “Edit->Changes->Show”, and also when saving document.

Problems

• Too Simple Data structure,which leads to only limitation capabilities supported
• Document Model modified during switching “Show” and “Hide” and saving modifications, which leads to stability problems. Especially, there are flying objects within the deletion records.

Design Proposals

In this chapter,we only present our imagine based on our investigated knowledge. Our knowledge should be limited. So, this is just the beginning and it is hard to avoid flaws.We open them, hope revisions and more creative ideas from senior volunteers.

Design Principles

1. Data Structure should meet the requirements of more powerful functionalities of Track Changes[extensibility,],for complete fidelity interoperability with MS Word
2. Document Model should not be changed when switching between different status
3. Avoid existing stability problems through the new design
4. Ensure compatibility with previous releases of Symphony and OpenOffice.org

Destination Data for Tracking

Plane Data

1. Characters
2. Paragraphs
3. Table
4. Section
5. Fields
6. Cross References
7. References
8. Footnote and Endnote
9. Comments

Attributes Data

1. Character Attributes
2. Paragraph Attributes
3. Numbering and Bullets
4. Breaks
5. Document Attributes

Format Data

1. Character Styles
2. Paragraph Styles
3. List Styles
4. Page Attributes
5. Page Styles
6. Drawing/Graphic/Textbox/OLE/...

Records for Tracking

1. Create and Delete Plane Data [Including Ctrl C/V/X]
2. Modify Attributes Data
3. Modify Format Data,among with which Drawing/Graphic/Textbox/OLE/... will be optional

Operations for Records

1. Accept
2. Reject
3. Combine
4. Split
5. Show Original
6. Show Final [Current Hide]
7. Show Original with Markup
8. Show Final with Markup [Current Show]

Principles of Combine and Split

Here,we only present simple principles. More creative input are welcome and expected.

Principles of Combine

1. Time-slice consideration
2. Range consideration

Principles of Split

1. Attribute Types,just like portion
2. Range consideration

Relation and Change Notification

From the relation analysis, the record will not only act as the data(SwModify),but also as the observer(SwClient). According to the current mechanism, the idea design will compromised for the technical limitation.

Destination Data and Record

1. When destination data is changed,the recorded data in the redline will be changed at the same time.
2. When switching between the show status, the status type of a record will be changed, the change will be notified to the destination data automatically. At that time, the destination data acts as an observer. Obviously, the record acts as the data model.

Record and Record

1. If there is only parallel relation between two records,their changes will not impact each other.
2. If there is nested relation between two records, when the outer record is accepted or rejected,the nested record should be notified. So, the record acts as an observer.

New Design

The new design is mainly re-using the mechanism of SwClient and SwModify, which represents the “Observer Design Patten”. And the design proposals should not limited to the idea, and they are open for discussion and creative brainstorm.

Data Structure

This is the class diagram draft of the new design. The middle class SwRedline_New will replace the currently existing class SwRedline. The left will be designed to keep compatible with the previous releases of Symphony and OpenOffice.org. And The right will be designed to support more complex contents based on the known mechanism. Among them, SwRedline_New will provide the basic abstract methods to operate the change records,whose implementations will assigned to the child class SwPaMRedline and SwFmtRedline to provide respectively.

Framework

For Plane Data

Data Design:

To plane data, we will align the design to the current implementation..Please refer to the left of the class diagram draft of new design. The goal of the design is to keep compatible with the previous releases of Symphony and OpenOffice.org. The redline will record the created characters or paragraphs and deleted characters or paragraphs.

Implementation Description:

1. Because the record contains the node index range, and content index range, once the destination data is changed,such as create new characters or paragraphs,the record will be updated along with the change.
2. If the record is operated, especially switching between “Show” and “Hide”, the redline list will be went through to set the related control variable or status of paragraph.
3. Presentation module will control the rendering of the change records at their original places,which mainly focus on building portions.
4. The change records will not be moved as before.

For Attributes Data

To attributes data, the attributes of characters will be separated. Its design and implementation will keep align with the original, which is mainly for the consideration of compatibility with previous releases. But, actually, only the position or range of the contents that the attributes applied are recorded, not the attributes. So,the characters attributes have also problem. So,this chapter will only focus on the paragraphs attributes. From the working logic of paragraphs attributes, there are two ways to apply paragraphs attributes. One is the SwItemSet,the other is SwTxtFmtColl,paragraph style. Please refer to the class diagram.

When the paragraph attribute is changed from the “Paragraph Properties” dialog,the pAttrSet member variable will be instanced,and the changed attributes will be stored in the member variable...By default, the pAttrSet is null.

Data Design:

To this part,the right of the class diagram draft of new design will be taken. The redline will record the changes of paragraph's pAttrSet, and also the changes of its parent because we see there is existing the inheritable relationship between SfxItemSet. Idealistically, we need the class of change record is able to take two roles, data and observer. Currently, according to the design and implementation of SwModify and SwClient, and their applications, the two roles can not be acted at the same time...Based on the current knowledge, the change record will only take the observer role in our new design.

Implementation Description:

1. Because the pAttrSet member variable of a paragraph will be changed if any paragraph attribute modification,so it is impossible to observe the pointer of pAttrSet. Thus, SwTxtNode will be observed by the record. Once any attribute modified of a paragraph, the existing record will be notified for ItemSet Update.
2. If the record is operated, especially switching between “Show” and “Hide”, the redline list will be went through to set the related control variable or status of SwTxtNode.
3. Presentation module will control the rendering of the change records at their original places,which mainly focus on building portions.
4. The change records will not be moved as before.

For Format Data

Almost, all the format data are child class of SwFmt. And SwFmt is a child class of SwModify. Thus,the main idea of the design is to use the current mechanism of SwModify and SwClient. Any change of SwModify can be notified to SwClient automatically. Conversely, we can also use SwClient to do certain operations of SwModify. Such as, when switching status,especially, switching status between “Show” and “Hide”, we can take use of the record to access instances of SwModify's child classes and set the pre-designed control attributes.

To Character Styles:

The character styles will be separated. Its design and implementation will keep align with the original, which is mainly for the consideration of compatibility with previous releases. From the class diagram, we can see the working logic and manage of character attributes are very complex. Currently, this part will not changed.

To Drawing/Graphic/Textbox/OLE/...:

Each flying object in Writer will have a kind of instance of SwFmt's child class. Please refer to the class diagram.

Paragraph Styles:

This is the diagram of paragraph styles.

In this chapter,we will use paragraph style as the example.

Data Design:

To this part,the right of the class diagram draft of new design will be taken. The redline will record the changes of SwFmt's aSet, and also the changes of its parent because we see there is existing the inheritable relationship between SfxItemSet. Moreover, the parent SwFmt's aSet is also under record because of the inheritable styles.

Implementation Description:

1. Because the aSet member variable of a SwFmt will be changed if any attribute modification,so it is impossible to observe the aSet. Thus, the instance of SwFmt's child class will be observed by the record. Once any attribute modified, the existing record will be notified for ItemSet Update.
2. If the record is operated, especially switching between “Show” and “Hide”, the redline list will be went through to set the related control variable or status of the instance of SwFmt's child class.
3. Presentation module will control the rendering of the change records at their original places,which mainly focus on building portions.
4. The change records will not be moved as before.

To Do Tasks

This is just a prototype of conception evaluation.

1.Make the Data Model Change to Support Missed Changes Types:

A)Paragraph Attributes :

• Data Model Design
• Data Structure Implementation
• Create Redline to Record the Limited Types' Paragraph Attributes in Memory
• ODT Export/Import
• Display as Comments //Not Include the Flying Object within the Recorded Area

B)Page Attributes : Total //The Effort Is Based on Situation after A) Delivered

• Data Model Design
• Data Structure Implementation
• Create Redline to Record Limited Types' Page Properties in Memory
• ODT Export/Import
• Display as Comments

C)Document Attributes //The Effort Is Based on Situation after A)&B) Delivered

• Data Model Design
• Data Structure Implementation
• Create Redline to Record Limited Types' Document Attributes
• ODT Export/Import
• Display as Comments

2. Support Basic Operations on Change Revision Records (Show/Hide/Show as Comments/Chow Original/Show Final),the Switch Will Not Impact Date Model.

A)Switch Between Hide and Show as Comments :

• Based on the Existing Design
• Based on the New Data Model

B)Show Original

C)Show Final

3. Compatibility with Previous Release

A)Import Old Version's Document and Map to New Data Model on Existing Capabilities

B)Keep the Same Result to Export the Data Part Same with Previous Release in the New Data Model

4. Make the Operations on Data Module to Fit The New Data Model

A)Accepting Changes

B)Rejecting Changes

5. Undo/Redo Support to Meet the New Data Model

A)Clear Old Logic

B)Enable New Implementation

Impact Areas and Risk

1. Almost the whole Writer component will be impacted
2. Rish is very high
3. Limitation of the Mechanism to distinguish a paragraph and the whole contents of a paragraph

Plan

Firstly, provide the capability to record paragraph attribute changes,interoperability with MS Word 2003 and capability with previous releases of Symphony/AOO

Then,according to the results of the first step, adjust the design as necessary to support flying objects and page attributes.

Appendix:

Some "meta" requirements for change tracking collected by ODF TC: http://wiki.oasis-open.org/office/Change Tracking Meta Requirements