Skip to content

Commit

Permalink
Merge branch 'develop'
Browse files Browse the repository at this point in the history
  • Loading branch information
JohnMcLear committed Nov 20, 2021
2 parents 571b37b + 2e19087 commit 868c685
Show file tree
Hide file tree
Showing 116 changed files with 6,600 additions and 6,498 deletions.
3 changes: 3 additions & 0 deletions .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
IMPORTANT: Please disable plugins prior to posting a bug report. If you have a problem with a plugin please post on the plugin repository. Thanks!

---
name: Bug report
about: Create a report to help us improve
Expand Down Expand Up @@ -28,6 +30,7 @@ If applicable, add screenshots to help explain your problem.
- OS: [e.g., Ubuntu 20.04]
- Node.js version (`node --version`):
- npm version (`npm --version`):
- Is the server free of plugins:

**Desktop (please complete the following information):**
- OS: [e.g. iOS]
Expand Down
32 changes: 8 additions & 24 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -1,29 +1,13 @@
<!--
Some key notes before you open a PR:
1. If you haven't already, please read https://github.com/ether/etherpad-lite/blob/master/CONTRIBUTING.md#pull-requests .
2. Run all the tests, both front-end and back-end. (see https://github.com/ether/etherpad-lite/blob/master/CONTRIBUTING.md#testing)
3. Keep business logic and validation on the server-side.
4. Update documentation.
5. Write `fixes #XXXX` in your comment to auto-close an issue.
1. Select which branch should this PR be merged in? By default, you should always merge to the develop branch.
2. PR name follows [convention](http://karma-runner.github.io/4.0/dev/git-commit-msg.html)
3. All tests pass locally, UI and Unit tests
4. All business logic and validations must be on the server-side
5. Update necessary Documentation
6. Put `closes #XXXX` in your comment to auto-close the issue that your PR fixes
Also, if you're new here
- Contribution Guide => https://github.com/ether/etherpad-lite/blob/master/CONTRIBUTING.md
If you're making a big change, please explain what problem it solves:
- Explain the purpose of the change. When adding a way to do X, explain why it is important to be able to do X.
- Show the current vs desired behavior with screenshots/GIFs.
-->

> Please provide enough information so that others can review your pull request:
<!-- You can skip this if you're fixing a typo or updating existing documentation -->

> Explain the **details** for making this change. What existing problem does the pull request solve?
<!-- Example: When "Adding a function to do X", explain why it is necessary to have a way to do X. -->

> Screenshots/GIFs
<!-- Add images/recordings to better visualize the change: expected/current behviour -->
8 changes: 2 additions & 6 deletions .github/workflows/backend-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -116,9 +116,7 @@ jobs:
node-version: 12

- name: Install all dependencies and symlink for ep_etherpad-lite
run: |
cd src
npm ci --no-optional
run: src/bin/installOnWindows.bat

- name: Fix up the settings.json
run: |
Expand Down Expand Up @@ -172,9 +170,7 @@ jobs:
# if npm correctly hoists the dependencies, the hoisting seems to confuse
# tools such as `npm outdated`, `npm update`, and some ESLint rules.
- name: Install all dependencies and symlink for ep_etherpad-lite
run: |
cd src
npm ci --no-optional
run: src/bin/installOnWindows.bat

- name: Fix up the settings.json
run: |
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/frontend-admin-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,9 @@ jobs:
- name: Write custom settings.json that enables the Admin UI tests
run: "sed -i 's/\"enableAdminUITests\": false/\"enableAdminUITests\": true,\\n\"users\":{\"admin\":{\"password\":\"changeme\",\"is_admin\":true}}/' settings.json"

- name: increase maxHttpBufferSize
run: "sed -i 's/\"maxHttpBufferSize\": 10000/\"maxHttpBufferSize\": 100000/' settings.json"

- name: Remove standard frontend test files, so only admin tests are run
run: mv src/tests/frontend/specs/* /tmp && mv /tmp/admin*.js src/tests/frontend/specs

Expand Down
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@ var/dirty.db
*.patch
npm-debug.log
*.DS_Store
.ep_initialized
*.crt
*.key
credentials.json
Expand Down
76 changes: 76 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,79 @@
# 1.8.15

### Security fixes

* Fixed leak of the writable pad ID when exporting from the pad's read-only ID.
This only matters if you treat the writeable pad IDs as secret (e.g., you are
not using [ep_padlist2](https://www.npmjs.com/package/ep_padlist2)) and you
share the pad's read-only ID with untrusted users. Instead of treating
writeable pad IDs as secret, you are encouraged to take advantage of
Etherpad's authentication and authorization mechanisms (e.g., use
[ep_openid_connect](https://www.npmjs.com/package/ep_openid_connect) with
[ep_readonly_guest](https://www.npmjs.com/package/ep_readonly_guest), or write
your own
[authentication](https://etherpad.org/doc/v1.8.14/#index_authenticate) and
[authorization](https://etherpad.org/doc/v1.8.14/#index_authorize) plugins).

### Compatibility changes

* The `logconfig` setting is deprecated.
* For plugin authors:
* Etherpad now uses [jsdom](https://github.com/jsdom/jsdom) instead of
[cheerio](https://cheerio.js.org/) for processing HTML imports. There are
two consequences of this change:
* `require('ep_etherpad-lite/node_modules/cheerio')` no longer works. To
fix, your plugin should directly depend on `cheerio` and do
`require('cheerio')`.
* The `node` context argument passed to the `collectContentImage` hook is
now an
[`HTMLImageElement`](https://developer.mozilla.org/en-US/docs/Web/API/HTMLImageElement)
object rather than a Cheerio Node-like object, so the API is slightly
different. See
[citizenos/ep_image_upload#49](https://github.com/citizenos/ep_image_upload/pull/49)
for an example fix.
* The `clientReady` server-side hook is deprecated; use the new `userJoin`
hook instead.
* The `init_<pluginName>` server-side hooks are now run every time Etherpad
starts up, not just the first time after the named plugin is installed.
* The `userLeave` server-side hook's context properties have changed:
* `auth`: Deprecated.
* `author`: Deprecated; use the new `authorId` property instead.
* `readonly`: Deprecated; use the new `readOnly` property instead.
* `rev`: Deprecated.
* Changes to the `src/static/js/Changeset.js` library:
* `opIterator()`: The unused start index parameter has been removed, as has
the unused `lastIndex()` method on the returned object.
* `smartOpAssembler()`: The returned object's `appendOpWithText()` method is
deprecated without a replacement available to plugins (if you need one,
let us know and we can make the private `opsFromText()` function public).
* Several functions that should have never been public are no longer
exported: `applyZip()`, `assert()`, `clearOp()`, `cloneOp()`, `copyOp()`,
`error()`, `followAttributes()`, `opString()`, `stringOp()`,
`textLinesMutator()`, `toBaseTen()`, `toSplices()`.

### Notable enhancements

* Simplified pad reload after importing an `.etherpad` file.
* For plugin authors:
* `clientVars` was added to the context for the `postAceInit` client-side
hook. Plugins should use this instead of the `clientVars` global variable.
* New `userJoin` server-side hook.
* The `userLeave` server-side hook has a new `socket` context property.
* The `helper.aNewPad()` function (accessible to client-side tests) now
accepts hook functions to inject when opening a pad. This can be used to
test any new client-side hooks your plugin provides.
* Chat improvements:
* The `chatNewMessage` client-side hook context has new properties:
* `message`: Provides access to the raw message object so that plugins can
see the original unprocessed message text and any added metadata.
* `rendered`: Allows plugins to completely override how the message is
rendered in the UI.
* New `chatSendMessage` client-side hook that enables plugins to process the
text before sending it to the server or augment the message object with
custom metadata.
* New `chatNewMessage` server-side hook to process new chat messages before
they are saved to the database and relayed to users.

# 1.8.14

### Security fixes
Expand Down
14 changes: 11 additions & 3 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,17 @@
number of the issue that is being fixed, in the form: Fixes #someIssueNumber
```
* if the PR is a **bug fix**:
* the first commit in the series must be a test that shows the failure
* subsequent commits will fix the bug and make the test pass
* the final commit message should include the text `Fixes: #xxx` to link it to its bug report
* The commit that fixes the bug should **include a regression test** that
would fail if the bug fix was reverted. Adding the regression test in the
same commit as the bug fix makes it easier for a reviewer to verify that the
test is appropriate for the bug fix.
* If there is a bug report, **the pull request description should include the
text "`Fixes #xxx`"** so that the bug report is auto-closed when the PR is
merged. It is less useful to say the same thing in a commit message because
GitHub will spam the bug report every time the commit is rebased, and
because a bug number alone becomes meaningless in forks. (A full URL would
be better, but ideally each commit is readable on its own without the need
to examine an external reference to understand motivation or context.)
* think about stability: code has to be backwards compatible as much as possible. Always **assume your code will be run with an older version of the DB/config file**
* if you want to remove a feature, **deprecate it instead**:
* write an issue with your deprecation plan
Expand Down
3 changes: 3 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ RUN export DEBIAN_FRONTEND=noninteractive; \
apt-get -qq --no-install-recommends install \
ca-certificates \
git \
curl \
${INSTALL_ABIWORD:+abiword} \
${INSTALL_SOFFICE:+libreoffice} \
&& \
Expand Down Expand Up @@ -94,5 +95,7 @@ COPY --chown=etherpad:etherpad ./settings.json.docker "${EP_DIR}"/settings.json
# Fix group permissions
RUN chmod -R g=u .

HEALTHCHECK --interval=20s --timeout=3s CMD curl -f http://localhost:9001 || exit 1

EXPOSE 9001
CMD ["node", "src/node/server.js"]
162 changes: 25 additions & 137 deletions doc/api/changeset_library.md
Original file line number Diff line number Diff line change
@@ -1,156 +1,44 @@
# Changeset Library

```
"Z:z>1|2=m=b*0|1+1$\n"
```

This is a Changeset. It's just a string and it's very difficult to read in this form. But the Changeset Library gives us some tools to read it.

A changeset describes the diff between two revisions of the document. The Browser sends changesets to the server and the server sends them to the clients to update them. These Changesets also get saved into the history of a pad. This allows us to go back to every revision from the past.
The [changeset
library](https://github.com/ether/etherpad-lite/blob/develop/src/static/js/Changeset.js)
provides tools to create, read, and apply changesets.

## Changeset.unpack(changeset)

* `changeset` {String}

This function returns an object representation of the changeset, similar to this:
## Changeset

```javascript
const Changeset = require('ep_etherpad-lite/static/js/Changeset');
```
{ oldLen: 35, newLen: 36, ops: '|2=m=b*0|1+1', charBank: '\n' }
```

* `oldLen` {Number} the original length of the document.
* `newLen` {Number} the length of the document after the changeset is applied.
* `ops` {String} the actual changes, introduced by this changeset.
* `charBank` {String} All characters that are added by this changeset.

## Changeset.opIterator(ops)
A changeset describes the difference between two revisions of a document. When a
user edits a pad, the browser generates and sends a changeset to the server,
which relays it to the other users and saves a copy (so that every past revision
is accessible).

* `ops` {String} The operators, returned by `Changeset.unpack()`
A transmitted changeset looks like this:

Returns an operator iterator. This iterator allows us to iterate over all operators that are in the changeset.

You can iterate with an opIterator using its `next()` and `hasNext()` methods. Next returns the `next()` operator object and `hasNext()` indicates, whether there are any operators left.

## The Operator object
There are 3 types of operators: `+`,`-` and `=`. These operators describe different changes to the document, beginning with the first character of the document. A `=` operator doesn't change the text, but it may add or remove text attributes. A `-` operator removes text. And a `+` Operator adds text and optionally adds some attributes to it.

* `opcode` {String} the operator type
* `chars` {Number} the length of the text changed by this operator.
* `lines` {Number} the number of lines changed by this operator.
* `attribs` {attribs} attributes set on this text.

### Example
```
{ opcode: '+',
chars: 1,
lines: 1,
attribs: '*0' }
'Z:z>1|2=m=b*0|1+1$\n'
```

## APool
## Attribute Pool

```javascript
const AttributePool = require('ep_etherpad-lite/static/js/AttributePool');
```
> var AttributePoolFactory = require("./utils/AttributePoolFactory");
> var apool = AttributePoolFactory.createAttributePool();
> console.log(apool)
{ numToAttrib: {},
attribToNum: {},
nextNum: 0,
putAttrib: [Function],
getAttrib: [Function],
getAttribKey: [Function],
getAttribValue: [Function],
eachAttrib: [Function],
toJsonable: [Function],
fromJsonable: [Function] }
```

This creates an empty apool. An apool saves which attributes were used during the history of a pad. There is one apool for each pad. It only saves the attributes that were really used, it doesn't save unused attributes. Let's fill this apool with some values

```
> apool.fromJsonable({"numToAttrib":{"0":["author","a.kVnWeomPADAT2pn9"],"1":["bold","true"],"2":["italic","true"]},"nextNum":3});
> console.log(apool)
{ numToAttrib:
{ '0': [ 'author', 'a.kVnWeomPADAT2pn9' ],
'1': [ 'bold', 'true' ],
'2': [ 'italic', 'true' ] },
attribToNum:
{ 'author,a.kVnWeomPADAT2pn9': 0,
'bold,true': 1,
'italic,true': 2 },
nextNum: 3,
putAttrib: [Function],
getAttrib: [Function],
getAttribKey: [Function],
getAttribValue: [Function],
eachAttrib: [Function],
toJsonable: [Function],
fromJsonable: [Function] }
```

We used the fromJsonable function to fill the empty apool with values. the fromJsonable and toJsonable functions are used to serialize and deserialize an apool. You can see that it stores the relation between numbers and attributes. So for example the attribute 1 is the attribute bold and vise versa. An attribute is always a key value pair. For stuff like bold and italic it's just 'italic':'true'. For authors it's author:$AUTHORID. So a character can be bold and italic. But it can't belong to multiple authors

```
> apool.getAttrib(1)
[ 'bold', 'true' ]
```

Simple example of how to get the key value pair for the attribute 1

## AText

```
> var atext = {"text":"bold text\nitalic text\nnormal text\n\n","attribs":"*0*1+9*0|1+1*0*1*2+b|1+1*0+b|2+2"};
> console.log(atext)
{ text: 'bold text\nitalic text\nnormal text\n\n',
attribs: '*0*1+9*0|1+1*0*1*2+b|1+1*0+b|2+2' }
```

This is an atext. An atext has two parts: text and attribs. The text is just the text of the pad as a string. We will look closer at the attribs at the next steps

```
> var opiterator = Changeset.opIterator(atext.attribs)
> console.log(opiterator)
{ next: [Function: next],
hasNext: [Function: hasNext],
lastIndex: [Function: lastIndex] }
> opiterator.next()
{ opcode: '+',
chars: 9,
lines: 0,
attribs: '*0*1' }
> opiterator.next()
{ opcode: '+',
chars: 1,
lines: 1,
attribs: '*0' }
> opiterator.next()
{ opcode: '+',
chars: 11,
lines: 0,
attribs: '*0*1*2' }
> opiterator.next()
{ opcode: '+',
chars: 1,
lines: 1,
attribs: '' }
> opiterator.next()
{ opcode: '+',
chars: 11,
lines: 0,
attribs: '*0' }
> opiterator.next()
{ opcode: '+',
chars: 2,
lines: 2,
attribs: '' }
```
Changesets do not include any attribute key–value pairs. Instead, they use
numeric identifiers that reference attributes kept in an [attribute
pool](https://github.com/ether/etherpad-lite/blob/develop/src/static/js/AttributePool.js).
This attribute interning reduces the transmission overhead of attributes that
are used many times.

The attribs are again a bunch of operators like .ops in the changeset was. But these operators are only + operators. They describe which part of the text has which attributes
There is one attribute pool per pad, and it includes every current and
historical attribute used in the pad.

## Resources / further reading
## Further Reading

Detailed information about the changesets & Easysync protocol:

* Easysync Protocol - [/doc/easysync/easysync-notes.pdf](https://github.com/ether/etherpad-lite/blob/develop/doc/easysync/easysync-notes.pdf)
* Etherpad and EasySync Technical Manual - [/doc/easysync/easysync-full-description.pdf](https://github.com/ether/etherpad-lite/blob/develop/doc/easysync/easysync-full-description.pdf)
* [Easysync Protocol](https://github.com/ether/etherpad-lite/blob/develop/doc/easysync/easysync-notes.pdf)
* [Etherpad and EasySync Technical Manual](https://github.com/ether/etherpad-lite/blob/develop/doc/easysync/easysync-full-description.pdf)
2 changes: 1 addition & 1 deletion doc/api/editbar.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ src/static/js/pad_editbar.js

## disable()

## toggleDropDown(dropdown, callback)
## toggleDropDown(dropdown)
Shows the dropdown `div.popup` whose `id` equals `dropdown`.

## registerCommand(cmd, callback)
Expand Down
Loading

0 comments on commit 868c685

Please sign in to comment.