Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for e2e key backups #736

Merged
merged 85 commits into from
Nov 21, 2018
Merged
Show file tree
Hide file tree
Changes from 75 commits
Commits
Show all changes
85 commits
Select commit Hold shift + click to select a range
fb1b554
initial pseudocode WIP for e2e online backups
ara4n Jan 15, 2018
e0c9b99
blindly move crypto.suggestKeyRestore over to /sync
ara4n Jan 18, 2018
69204d4
Merge branch 'develop' into matthew/e2e_backups
ara4n May 27, 2018
d556189
initial implementation of e2e key backup and restore
uhoreg Aug 8, 2018
1faf477
fix formatting and fix authedRequest usage
uhoreg Aug 23, 2018
fb8efe3
initial draft of API for working with backup versions
uhoreg Aug 23, 2018
75107f9
pass in key rather than decryption object to restoreKeyBackups
uhoreg Aug 23, 2018
e5ec479
check that crypto is enabled
uhoreg Aug 23, 2018
73e294b
add copyright header to backup.spec
uhoreg Aug 23, 2018
ec5fff2
Merge branch 'e2e_backups' of git://github.com/uhoreg/matrix-js-sdk i…
dbkr Aug 24, 2018
017f81e
fix some bugs
uhoreg Aug 24, 2018
bf873bd
split the backup version creation into two different methods
uhoreg Aug 25, 2018
29db856
Merge branch 'e2e_backups' of git://github.com/uhoreg/matrix-js-sdk i…
dbkr Sep 11, 2018
72bd51f
Merge remote-tracking branch 'origin/develop' into uhoreg-e2e_backups
dbkr Sep 11, 2018
3838fab
WIP e2e key backup support
dbkr Sep 13, 2018
e789747
Check sigs on e2e backup & enable it if we can
dbkr Sep 14, 2018
073fb73
Make multi-room key restore work
dbkr Sep 17, 2018
009430e
Add isValidRecoveryKey
dbkr Sep 17, 2018
f75d188
Soe progress on linting
dbkr Sep 17, 2018
3af9af9
More linting
dbkr Sep 17, 2018
54c443a
Make tests pass
dbkr Sep 18, 2018
e4bb37b
Fix lint mostly
dbkr Sep 18, 2018
0bad7b2
Fix lint
dbkr Sep 18, 2018
a78825e
Bump to Olm 2.3.0 for PkEncryption
dbkr Sep 18, 2018
1b62a21
Free PkEncryption/Decryption objects
dbkr Sep 18, 2018
2f4c1df
Test all 3 code paths on backup restore
dbkr Sep 18, 2018
e9b0aca
Merge remote-tracking branch 'origin/develop' into dbkr/e2e_backups
dbkr Oct 2, 2018
ce2058a
Merge branch 'dbkr/wasm' into dbkr/e2e_backups
dbkr Oct 2, 2018
7cd101d
Fix recovery key format
dbkr Oct 2, 2018
262ace1
commit the recovery key util file
dbkr Oct 3, 2018
258adda
retry key backups when they fail
uhoreg Oct 4, 2018
89c3f6f
Merge remote-tracking branch 'origin/develop' into dbkr/e2e_backups
dbkr Oct 5, 2018
b3fe05e
Merge remote-tracking branch 'origin/develop' into dbkr/e2e_backups
dbkr Oct 9, 2018
59e6066
Replace base58check with a simple parity check
dbkr Oct 9, 2018
ada4b6e
Lint
dbkr Oct 9, 2018
da65f43
wrap backup sending in a try, and add delays
uhoreg Oct 10, 2018
fc59bc2
add localstorage support for key backups
uhoreg Oct 10, 2018
3957006
Merge remote-tracking branch 'upstream/dbkr/e2e_backups' into e2e_bac…
uhoreg Oct 11, 2018
9b12c22
de-lint plus some minor fixes
uhoreg Oct 12, 2018
91fb7b0
fix unit tests for backup recovery
uhoreg Oct 12, 2018
d49c0a1
more de-linting and fixing
uhoreg Oct 12, 2018
40d0a82
remove accidental change to eslintrc
uhoreg Oct 12, 2018
434ac86
properly fill out the is_verified and first_message_index fields
uhoreg Oct 19, 2018
322ef1f
update backup algorithm name to agree with the proposal
uhoreg Oct 22, 2018
f165b55
Merge branch 'e2e_backups' of git://github.com/uhoreg/matrix-js-sdk i…
dbkr Oct 24, 2018
40cb37e
Update to Olm 3
dbkr Oct 24, 2018
5e8061f
Merge remote-tracking branch 'origin/develop' into dbkr/e2e_backups
dbkr Oct 25, 2018
243bab7
Merge branch 'dbkr/stop_devicelist' into dbkr/e2e_backups
dbkr Oct 25, 2018
b3bb99d
Stop client after backup tests
dbkr Oct 25, 2018
a6bf40d
We can always import these now
dbkr Oct 25, 2018
0e26247
Speed up time rather than increasing timeouts
dbkr Oct 26, 2018
6518bff
Merge remote-tracking branch 'origin/develop' into dbkr/e2e_backups
dbkr Oct 26, 2018
563e6b3
Fix jsdoc
dbkr Oct 26, 2018
3b2f2f9
Bump db version
dbkr Oct 26, 2018
e51d2dd
Fix a few e2e backup bits
dbkr Oct 30, 2018
a2430db
Fix DeviceList index of users by identity key
dbkr Oct 30, 2018
2814932
lint
dbkr Oct 30, 2018
8ab84de
PR feedback 1/n
dbkr Oct 31, 2018
c6ad066
factor out duplicated test code
dbkr Oct 31, 2018
2b46c56
Add crypto. prefix to keyBackupStatus event
dbkr Oct 31, 2018
c5e7bed
Conclusion: no, it shouldn't
dbkr Oct 31, 2018
f5846b8
More modern loop syntax
dbkr Oct 31, 2018
6de2134
Change getDeviceByIdentityKey() to just the 2 arg version
dbkr Oct 31, 2018
5c5ce0d
Typo
dbkr Oct 31, 2018
db2897c
Remove spurious interlopers
dbkr Oct 31, 2018
c77ecad
clarify comment
dbkr Oct 31, 2018
7c0b910
remove unnecessary isFinite check
dbkr Oct 31, 2018
63e9f79
Remove unnecessary if
dbkr Oct 31, 2018
2f219f8
Catch exceptions from backupGroupSession()
dbkr Oct 31, 2018
5e98859
random double linebreak
dbkr Oct 31, 2018
2af5643
Clarify comment
dbkr Oct 31, 2018
c7a0c14
refer to getAllEndToEndInboundGroupSessions for magic numbers
dbkr Oct 31, 2018
0477f35
Fix key forwarded count
dbkr Oct 31, 2018
29d92d3
Lint
dbkr Oct 31, 2018
379f290
Add package-lock.json
dbkr Nov 13, 2018
907cf19
Merge remote-tracking branch 'origin/develop' into dbkr/e2e_backups
dbkr Nov 15, 2018
c53c6a9
Update package-lock
dbkr Nov 15, 2018
d99a22d
Update to new API
dbkr Nov 16, 2018
44d9927
Support passphrase-based e2e key backups
dbkr Nov 20, 2018
cb51799
Make backup restore work
dbkr Nov 20, 2018
6047838
lint
dbkr Nov 20, 2018
eeea706
Add randomString factored out from client secret
dbkr Nov 20, 2018
abd2ac7
Rename backup API call in test
dbkr Nov 20, 2018
092f421
docs
dbkr Nov 21, 2018
bd2cf18
Merge pull request #786 from matrix-org/dbkr/e2e_backups_passphrase
dbkr Nov 21, 2018
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6,856 changes: 6,856 additions & 0 deletions package-lock.json

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@
"babel-runtime": "^6.26.0",
"bluebird": "^3.5.0",
"browser-request": "^0.3.3",
"bs58": "^4.0.1",
"content-type": "^1.0.2",
"loglevel": "1.6.1",
"request": "^2.53.0"
Expand Down
470 changes: 470 additions & 0 deletions spec/unit/crypto/backup.spec.js

Large diffs are not rendered by default.

348 changes: 347 additions & 1 deletion src/client.js
Original file line number Diff line number Diff line change
Expand Up @@ -41,12 +41,14 @@ const SyncApi = require("./sync");
const MatrixBaseApis = require("./base-apis");
const MatrixError = httpApi.MatrixError;
const ContentHelpers = require("./content-helpers");
const olmlib = require("./crypto/olmlib");

import ReEmitter from './ReEmitter';
import RoomList from './crypto/RoomList';

import Crypto from './crypto';
import { isCryptoAvailable } from './crypto';
import { encodeRecoveryKey, decodeRecoveryKey } from './crypto/recoverykey';

// Disable warnings for now: we use deprecated bluebird functions
// and need to migrate, but they spam the console with warnings.
Expand All @@ -56,6 +58,29 @@ Promise.config({warnings: false});
const SCROLLBACK_DELAY_MS = 3000;
const CRYPTO_ENABLED = isCryptoAvailable();

function keysFromRecoverySession(sessions, decryptionKey, roomId) {
const keys = [];
for (const [sessionId, sessionData] of Object.entries(sessions)) {
try {
const decrypted = keyFromRecoverySession(sessionData, decryptionKey);
decrypted.session_id = sessionId;
decrypted.room_id = roomId;
keys.push(decrypted);
} catch (e) {
console.log("Failed to decrypt session from backup");
}
}
return keys;
}

function keyFromRecoverySession(session, decryptionKey) {
return JSON.parse(decryptionKey.decrypt(
session.session_data.ephemeral,
session.session_data.mac,
session.session_data.ciphertext,
));
}

/**
* Construct a Matrix Client. Only directly construct this if you want to use
* custom modules. Normally, {@link createClient} should be used
Expand Down Expand Up @@ -533,7 +558,15 @@ MatrixClient.prototype.setDeviceVerified = function(userId, deviceId, verified)
if (verified === undefined) {
verified = true;
}
return _setDeviceVerification(this, userId, deviceId, verified, null);
const prom = _setDeviceVerification(this, userId, deviceId, verified, null);

// if one of the user's own devices is being marked as verified / unverified,
// check the key backup status, since whether or not we use this depends on
// whether it has a signature from a verified device
if (userId == this.credentials.userId) {
this._crypto.checkKeyBackup();
}
return prom;
};

/**
Expand Down Expand Up @@ -737,6 +770,301 @@ MatrixClient.prototype.importRoomKeys = function(keys) {
return this._crypto.importRoomKeys(keys);
};

/**
* Get information about the current key backup.
* @returns {Promise} Information object from API or null
*/
MatrixClient.prototype.getKeyBackupVersion = function() {
return this._http.authedRequest(
undefined, "GET", "/room_keys/version",
).then((res) => {
if (res.algorithm !== olmlib.MEGOLM_BACKUP_ALGORITHM) {
const err = "Unknown backup algorithm: " + res.algorithm;
return Promise.reject(err);
} else if (!(typeof res.auth_data === "object")
|| !res.auth_data.public_key) {
const err = "Invalid backup data returned";
return Promise.reject(err);
} else {
return res;
}
}).catch((e) => {
if (e.errcode === 'M_NOT_FOUND') {
return null;
} else {
throw e;
}
});
};

/**
* @param {object} info key backup info dict from getKeyBackupVersion()
* @return {object} {
* usable: [bool], // is the backup trusted, true iff there is a sig that is valid & from a trusted device
* sigs: [
* valid: [bool],
* device: [DeviceInfo],
* ]
* }
*/
MatrixClient.prototype.isKeyBackupTrusted = function(info) {
return this._crypto.isKeyBackupTrusted(info);
};

/**
* @returns {bool} true if the client is configured to back up keys to
* the server, otherwise false.
*/
MatrixClient.prototype.getKeyBackupEnabled = function() {
if (this._crypto === null) {
throw new Error("End-to-end encryption disabled");
}
return Boolean(this._crypto.backupKey);
};

/**
* Enable backing up of keys, using data previously returned from
* getKeyBackupVersion.
*
* @param {object} info Backup information object as returned by getKeyBackupVersion
*/
MatrixClient.prototype.enableKeyBackup = function(info) {
if (this._crypto === null) {
throw new Error("End-to-end encryption disabled");
}

this._crypto.backupInfo = info;
if (this._crypto.backupKey) this._crypto.backupKey.free();
this._crypto.backupKey = new global.Olm.PkEncryption();
this._crypto.backupKey.set_recipient_key(info.auth_data.public_key);

this.emit('crypto.keyBackupStatus', true);
};

/**
* Disable backing up of keys.
*/
MatrixClient.prototype.disableKeyBackup = function() {
if (this._crypto === null) {
throw new Error("End-to-end encryption disabled");
}

this._crypto.backupInfo = null;
if (this._crypto.backupKey) this._crypto.backupKey.free();
this._crypto.backupKey = null;

this.emit('crypto.keyBackupStatus', false);
};

/**
* Set up the data required to create a new backup version. The backup version
* will not be created and enabled until createKeyBackupVersion is called.
*
* @returns {object} Object that can be passed to createKeyBackupVersion and
* additionally has a 'recovery_key' member with the user-facing recovery key string.
*/
MatrixClient.prototype.prepareKeyBackupVersion = function() {
if (this._crypto === null) {
throw new Error("End-to-end encryption disabled");
}

const decryption = new global.Olm.PkDecryption();
try {
const publicKey = decryption.generate_key();
return {
algorithm: olmlib.MEGOLM_BACKUP_ALGORITHM,
auth_data: {
public_key: publicKey,
},
recovery_key: encodeRecoveryKey(decryption.get_private_key()),
};
} finally {
decryption.free();
}
};

/**
* Create a new key backup version and enable it, using the information return
* from prepareKeyBackupVersion.
*
* @param {object} info Info object from prepareKeyBackupVersion
* @returns {Promise<object>} Object with 'version' param indicating the version created
*/
MatrixClient.prototype.createKeyBackupVersion = function(info) {
if (this._crypto === null) {
throw new Error("End-to-end encryption disabled");
}

const data = {
algorithm: info.algorithm,
auth_data: info.auth_data,
};
return this._crypto._signObject(data.auth_data).then(() => {
return this._http.authedRequest(
undefined, "POST", "/room_keys/version", undefined, data,
);
}).then((res) => {
this.enableKeyBackup({
algorithm: info.algorithm,
auth_data: info.auth_data,
version: res.version,
});
return res;
});
};

MatrixClient.prototype.deleteKeyBackupVersion = function(version) {
if (this._crypto === null) {
throw new Error("End-to-end encryption disabled");
}

// If we're currently backing up to this backup... stop.
// (We start using it automatically in createKeyBackupVersion
// so this is symmetrical).
if (this._crypto.backupInfo && this._crypto.backupInfo.version === version) {
this.disableKeyBackup();
}

const path = utils.encodeUri("/room_keys/version/$version", {
$version: version,
});

return this._http.authedRequest(
undefined, "DELETE", path, undefined, undefined,
);
};

MatrixClient.prototype._makeKeyBackupPath = function(roomId, sessionId, version) {
let path;
if (sessionId !== undefined) {
path = utils.encodeUri("/room_keys/keys/$roomId/$sessionId", {
$roomId: roomId,
$sessionId: sessionId,
});
} else if (roomId !== undefined) {
path = utils.encodeUri("/room_keys/keys/$roomId", {
$roomId: roomId,
});
} else {
path = "/room_keys/keys";
}
const queryData = version === undefined ? undefined : { version: version };
return {
path: path,
queryData: queryData,
};
};

/**
* Back up session keys to the homeserver.
* @param {string} roomId ID of the room that the keys are for Optional.
* @param {string} sessionId ID of the session that the keys are for Optional.
* @param {integer} version backup version Optional.
* @param {object} data Object keys to send
* @return {module:client.Promise} a promise that will resolve when the keys
* are uploaded
*/
MatrixClient.prototype.sendKeyBackup = function(roomId, sessionId, version, data) {
if (this._crypto === null) {
throw new Error("End-to-end encryption disabled");
}

const path = this._makeKeyBackupPath(roomId, sessionId, version);
return this._http.authedRequest(
undefined, "PUT", path.path, path.queryData, data,
);
};

MatrixClient.prototype.backupAllGroupSessions = function(version) {
if (this._crypto === null) {
throw new Error("End-to-end encryption disabled");
}

return this._crypto.backupAllGroupSessions(version);
};

MatrixClient.prototype.isValidRecoveryKey = function(recoveryKey) {
try {
decodeRecoveryKey(recoveryKey);
return true;
} catch (e) {
return false;
}
};

MatrixClient.prototype.restoreKeyBackups = function(
recoveryKey, targetRoomId, targetSessionId, version,
) {
if (this._crypto === null) {
throw new Error("End-to-end encryption disabled");
}
let totalKeyCount = 0;
let keys = [];

const path = this._makeKeyBackupPath(targetRoomId, targetSessionId, version);

// FIXME: see the FIXME in createKeyBackupVersion
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This FIXME seems to be obsolete, since I don't see any relevant FIXME in createKeyBackupVersion. (The one FIXME in there seems unrelated.)

const privkey = decodeRecoveryKey(recoveryKey);
const decryption = new global.Olm.PkDecryption();
try {
decryption.init_with_private_key(privkey);
} catch(e) {
decryption.free();
throw e;
}

return this._http.authedRequest(
undefined, "GET", path.path, path.queryData,
).then((res) => {
if (res.rooms) {
for (const [roomId, roomData] of Object.entries(res.rooms)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ooh, i had no idea you could do that

if (!roomData.sessions) continue;

totalKeyCount += Object.keys(roomData.sessions).length;
const roomKeys = keysFromRecoverySession(
roomData.sessions, decryption, roomId, roomKeys,
);
for (const k of roomKeys) {
k.room_id = roomId;
keys.push(k);
}
}
} else if (res.sessions) {
totalKeyCount = Object.keys(res.sessions).length;
keys = keysFromRecoverySession(
res.sessions, decryption, targetRoomId, keys,
);
} else {
totalKeyCount = 1;
try {
const key = keyFromRecoverySession(res, decryption);
key.room_id = targetRoomId;
key.session_id = targetSessionId;
keys.push(key);
} catch (e) {
console.log("Failed to decrypt session from backup");
}
}

return this.importRoomKeys(keys);
}).then(() => {
return {total: totalKeyCount, imported: keys.length};
}).finally(() => {
decryption.free();
});
};

MatrixClient.prototype.deleteKeysFromBackup = function(roomId, sessionId, version) {
if (this._crypto === null) {
throw new Error("End-to-end encryption disabled");
}

const path = this._makeKeyBackupPath(roomId, sessionId, version);
return this._http.authedRequest(
undefined, "DELETE", path.path, path.queryData,
);
};

// Group ops
// =========
// Operations on groups that come down the sync stream (ie. ones the
Expand Down Expand Up @@ -3735,6 +4063,24 @@ module.exports.CRYPTO_ENABLED = CRYPTO_ENABLED;
* });
*/

/**
* Fires whenever the status of e2e key backup changes, as returned by getKeyBackupEnabled()
* @event module:client~MatrixClient#"crypto.keyBackupStatus"
* @param {bool} enabled true if key backup has been enabled, otherwise false
* @example
* matrixClient.on("crypto.keyBackupStatus", function(enabled){
* if (enabled) {
* [...]
* }
* });
*/

/**
* Fires when we want to suggest to the user that they restore their megolm keys
* from backup or by cross-signing the device.
*
* @event module:client~MatrixClient#"crypto.suggestKeyRestore"
*/

// EventEmitter JSDocs

Expand Down
Loading