-
-
Notifications
You must be signed in to change notification settings - Fork 611
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support for e2e key backups #736
Merged
Merged
Changes from 75 commits
Commits
Show all changes
85 commits
Select commit
Hold shift + click to select a range
fb1b554
initial pseudocode WIP for e2e online backups
ara4n e0c9b99
blindly move crypto.suggestKeyRestore over to /sync
ara4n 69204d4
Merge branch 'develop' into matthew/e2e_backups
ara4n d556189
initial implementation of e2e key backup and restore
uhoreg 1faf477
fix formatting and fix authedRequest usage
uhoreg fb8efe3
initial draft of API for working with backup versions
uhoreg 75107f9
pass in key rather than decryption object to restoreKeyBackups
uhoreg e5ec479
check that crypto is enabled
uhoreg 73e294b
add copyright header to backup.spec
uhoreg ec5fff2
Merge branch 'e2e_backups' of git://github.com/uhoreg/matrix-js-sdk i…
dbkr 017f81e
fix some bugs
uhoreg bf873bd
split the backup version creation into two different methods
uhoreg 29db856
Merge branch 'e2e_backups' of git://github.com/uhoreg/matrix-js-sdk i…
dbkr 72bd51f
Merge remote-tracking branch 'origin/develop' into uhoreg-e2e_backups
dbkr 3838fab
WIP e2e key backup support
dbkr e789747
Check sigs on e2e backup & enable it if we can
dbkr 073fb73
Make multi-room key restore work
dbkr 009430e
Add isValidRecoveryKey
dbkr f75d188
Soe progress on linting
dbkr 3af9af9
More linting
dbkr 54c443a
Make tests pass
dbkr e4bb37b
Fix lint mostly
dbkr 0bad7b2
Fix lint
dbkr a78825e
Bump to Olm 2.3.0 for PkEncryption
dbkr 1b62a21
Free PkEncryption/Decryption objects
dbkr 2f4c1df
Test all 3 code paths on backup restore
dbkr e9b0aca
Merge remote-tracking branch 'origin/develop' into dbkr/e2e_backups
dbkr ce2058a
Merge branch 'dbkr/wasm' into dbkr/e2e_backups
dbkr 7cd101d
Fix recovery key format
dbkr 262ace1
commit the recovery key util file
dbkr 258adda
retry key backups when they fail
uhoreg 89c3f6f
Merge remote-tracking branch 'origin/develop' into dbkr/e2e_backups
dbkr b3fe05e
Merge remote-tracking branch 'origin/develop' into dbkr/e2e_backups
dbkr 59e6066
Replace base58check with a simple parity check
dbkr ada4b6e
Lint
dbkr da65f43
wrap backup sending in a try, and add delays
uhoreg fc59bc2
add localstorage support for key backups
uhoreg 3957006
Merge remote-tracking branch 'upstream/dbkr/e2e_backups' into e2e_bac…
uhoreg 9b12c22
de-lint plus some minor fixes
uhoreg 91fb7b0
fix unit tests for backup recovery
uhoreg d49c0a1
more de-linting and fixing
uhoreg 40d0a82
remove accidental change to eslintrc
uhoreg 434ac86
properly fill out the is_verified and first_message_index fields
uhoreg 322ef1f
update backup algorithm name to agree with the proposal
uhoreg f165b55
Merge branch 'e2e_backups' of git://github.com/uhoreg/matrix-js-sdk i…
dbkr 40cb37e
Update to Olm 3
dbkr 5e8061f
Merge remote-tracking branch 'origin/develop' into dbkr/e2e_backups
dbkr 243bab7
Merge branch 'dbkr/stop_devicelist' into dbkr/e2e_backups
dbkr b3bb99d
Stop client after backup tests
dbkr a6bf40d
We can always import these now
dbkr 0e26247
Speed up time rather than increasing timeouts
dbkr 6518bff
Merge remote-tracking branch 'origin/develop' into dbkr/e2e_backups
dbkr 563e6b3
Fix jsdoc
dbkr 3b2f2f9
Bump db version
dbkr e51d2dd
Fix a few e2e backup bits
dbkr a2430db
Fix DeviceList index of users by identity key
dbkr 2814932
lint
dbkr 8ab84de
PR feedback 1/n
dbkr c6ad066
factor out duplicated test code
dbkr 2b46c56
Add crypto. prefix to keyBackupStatus event
dbkr c5e7bed
Conclusion: no, it shouldn't
dbkr f5846b8
More modern loop syntax
dbkr 6de2134
Change getDeviceByIdentityKey() to just the 2 arg version
dbkr 5c5ce0d
Typo
dbkr db2897c
Remove spurious interlopers
dbkr c77ecad
clarify comment
dbkr 7c0b910
remove unnecessary isFinite check
dbkr 63e9f79
Remove unnecessary if
dbkr 2f219f8
Catch exceptions from backupGroupSession()
dbkr 5e98859
random double linebreak
dbkr 2af5643
Clarify comment
dbkr c7a0c14
refer to getAllEndToEndInboundGroupSessions for magic numbers
dbkr 0477f35
Fix key forwarded count
dbkr 29d92d3
Lint
dbkr 379f290
Add package-lock.json
dbkr 907cf19
Merge remote-tracking branch 'origin/develop' into dbkr/e2e_backups
dbkr c53c6a9
Update package-lock
dbkr d99a22d
Update to new API
dbkr 44d9927
Support passphrase-based e2e key backups
dbkr cb51799
Make backup restore work
dbkr 6047838
lint
dbkr eeea706
Add randomString factored out from client secret
dbkr abd2ac7
Rename backup API call in test
dbkr 092f421
docs
dbkr bd2cf18
Merge pull request #786 from matrix-org/dbkr/e2e_backups_passphrase
dbkr File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -41,12 +41,14 @@ const SyncApi = require("./sync"); | |
const MatrixBaseApis = require("./base-apis"); | ||
const MatrixError = httpApi.MatrixError; | ||
const ContentHelpers = require("./content-helpers"); | ||
const olmlib = require("./crypto/olmlib"); | ||
|
||
import ReEmitter from './ReEmitter'; | ||
import RoomList from './crypto/RoomList'; | ||
|
||
import Crypto from './crypto'; | ||
import { isCryptoAvailable } from './crypto'; | ||
import { encodeRecoveryKey, decodeRecoveryKey } from './crypto/recoverykey'; | ||
|
||
// Disable warnings for now: we use deprecated bluebird functions | ||
// and need to migrate, but they spam the console with warnings. | ||
|
@@ -56,6 +58,29 @@ Promise.config({warnings: false}); | |
const SCROLLBACK_DELAY_MS = 3000; | ||
const CRYPTO_ENABLED = isCryptoAvailable(); | ||
|
||
function keysFromRecoverySession(sessions, decryptionKey, roomId) { | ||
const keys = []; | ||
for (const [sessionId, sessionData] of Object.entries(sessions)) { | ||
try { | ||
const decrypted = keyFromRecoverySession(sessionData, decryptionKey); | ||
decrypted.session_id = sessionId; | ||
decrypted.room_id = roomId; | ||
keys.push(decrypted); | ||
} catch (e) { | ||
console.log("Failed to decrypt session from backup"); | ||
} | ||
} | ||
return keys; | ||
} | ||
|
||
function keyFromRecoverySession(session, decryptionKey) { | ||
return JSON.parse(decryptionKey.decrypt( | ||
session.session_data.ephemeral, | ||
session.session_data.mac, | ||
session.session_data.ciphertext, | ||
)); | ||
} | ||
|
||
/** | ||
* Construct a Matrix Client. Only directly construct this if you want to use | ||
* custom modules. Normally, {@link createClient} should be used | ||
|
@@ -533,7 +558,15 @@ MatrixClient.prototype.setDeviceVerified = function(userId, deviceId, verified) | |
if (verified === undefined) { | ||
verified = true; | ||
} | ||
return _setDeviceVerification(this, userId, deviceId, verified, null); | ||
const prom = _setDeviceVerification(this, userId, deviceId, verified, null); | ||
|
||
// if one of the user's own devices is being marked as verified / unverified, | ||
// check the key backup status, since whether or not we use this depends on | ||
// whether it has a signature from a verified device | ||
if (userId == this.credentials.userId) { | ||
this._crypto.checkKeyBackup(); | ||
} | ||
return prom; | ||
}; | ||
|
||
/** | ||
|
@@ -737,6 +770,301 @@ MatrixClient.prototype.importRoomKeys = function(keys) { | |
return this._crypto.importRoomKeys(keys); | ||
}; | ||
|
||
/** | ||
* Get information about the current key backup. | ||
* @returns {Promise} Information object from API or null | ||
*/ | ||
MatrixClient.prototype.getKeyBackupVersion = function() { | ||
return this._http.authedRequest( | ||
undefined, "GET", "/room_keys/version", | ||
).then((res) => { | ||
if (res.algorithm !== olmlib.MEGOLM_BACKUP_ALGORITHM) { | ||
const err = "Unknown backup algorithm: " + res.algorithm; | ||
return Promise.reject(err); | ||
} else if (!(typeof res.auth_data === "object") | ||
|| !res.auth_data.public_key) { | ||
const err = "Invalid backup data returned"; | ||
return Promise.reject(err); | ||
} else { | ||
return res; | ||
} | ||
}).catch((e) => { | ||
if (e.errcode === 'M_NOT_FOUND') { | ||
return null; | ||
} else { | ||
throw e; | ||
} | ||
}); | ||
}; | ||
|
||
/** | ||
* @param {object} info key backup info dict from getKeyBackupVersion() | ||
* @return {object} { | ||
* usable: [bool], // is the backup trusted, true iff there is a sig that is valid & from a trusted device | ||
* sigs: [ | ||
* valid: [bool], | ||
* device: [DeviceInfo], | ||
* ] | ||
* } | ||
*/ | ||
MatrixClient.prototype.isKeyBackupTrusted = function(info) { | ||
return this._crypto.isKeyBackupTrusted(info); | ||
}; | ||
|
||
/** | ||
* @returns {bool} true if the client is configured to back up keys to | ||
* the server, otherwise false. | ||
*/ | ||
MatrixClient.prototype.getKeyBackupEnabled = function() { | ||
if (this._crypto === null) { | ||
throw new Error("End-to-end encryption disabled"); | ||
} | ||
return Boolean(this._crypto.backupKey); | ||
}; | ||
|
||
/** | ||
* Enable backing up of keys, using data previously returned from | ||
* getKeyBackupVersion. | ||
* | ||
* @param {object} info Backup information object as returned by getKeyBackupVersion | ||
*/ | ||
MatrixClient.prototype.enableKeyBackup = function(info) { | ||
if (this._crypto === null) { | ||
throw new Error("End-to-end encryption disabled"); | ||
} | ||
|
||
this._crypto.backupInfo = info; | ||
if (this._crypto.backupKey) this._crypto.backupKey.free(); | ||
this._crypto.backupKey = new global.Olm.PkEncryption(); | ||
this._crypto.backupKey.set_recipient_key(info.auth_data.public_key); | ||
|
||
this.emit('crypto.keyBackupStatus', true); | ||
}; | ||
|
||
/** | ||
* Disable backing up of keys. | ||
*/ | ||
MatrixClient.prototype.disableKeyBackup = function() { | ||
if (this._crypto === null) { | ||
throw new Error("End-to-end encryption disabled"); | ||
} | ||
|
||
this._crypto.backupInfo = null; | ||
if (this._crypto.backupKey) this._crypto.backupKey.free(); | ||
this._crypto.backupKey = null; | ||
|
||
this.emit('crypto.keyBackupStatus', false); | ||
}; | ||
|
||
/** | ||
* Set up the data required to create a new backup version. The backup version | ||
* will not be created and enabled until createKeyBackupVersion is called. | ||
* | ||
* @returns {object} Object that can be passed to createKeyBackupVersion and | ||
* additionally has a 'recovery_key' member with the user-facing recovery key string. | ||
*/ | ||
MatrixClient.prototype.prepareKeyBackupVersion = function() { | ||
if (this._crypto === null) { | ||
throw new Error("End-to-end encryption disabled"); | ||
} | ||
|
||
const decryption = new global.Olm.PkDecryption(); | ||
try { | ||
const publicKey = decryption.generate_key(); | ||
return { | ||
algorithm: olmlib.MEGOLM_BACKUP_ALGORITHM, | ||
auth_data: { | ||
public_key: publicKey, | ||
}, | ||
recovery_key: encodeRecoveryKey(decryption.get_private_key()), | ||
}; | ||
} finally { | ||
decryption.free(); | ||
} | ||
}; | ||
|
||
/** | ||
* Create a new key backup version and enable it, using the information return | ||
* from prepareKeyBackupVersion. | ||
* | ||
* @param {object} info Info object from prepareKeyBackupVersion | ||
* @returns {Promise<object>} Object with 'version' param indicating the version created | ||
*/ | ||
MatrixClient.prototype.createKeyBackupVersion = function(info) { | ||
if (this._crypto === null) { | ||
throw new Error("End-to-end encryption disabled"); | ||
} | ||
|
||
const data = { | ||
algorithm: info.algorithm, | ||
auth_data: info.auth_data, | ||
}; | ||
return this._crypto._signObject(data.auth_data).then(() => { | ||
return this._http.authedRequest( | ||
undefined, "POST", "/room_keys/version", undefined, data, | ||
); | ||
}).then((res) => { | ||
this.enableKeyBackup({ | ||
algorithm: info.algorithm, | ||
auth_data: info.auth_data, | ||
version: res.version, | ||
}); | ||
return res; | ||
}); | ||
}; | ||
|
||
MatrixClient.prototype.deleteKeyBackupVersion = function(version) { | ||
if (this._crypto === null) { | ||
throw new Error("End-to-end encryption disabled"); | ||
} | ||
|
||
// If we're currently backing up to this backup... stop. | ||
// (We start using it automatically in createKeyBackupVersion | ||
// so this is symmetrical). | ||
if (this._crypto.backupInfo && this._crypto.backupInfo.version === version) { | ||
this.disableKeyBackup(); | ||
} | ||
|
||
const path = utils.encodeUri("/room_keys/version/$version", { | ||
$version: version, | ||
}); | ||
|
||
return this._http.authedRequest( | ||
undefined, "DELETE", path, undefined, undefined, | ||
); | ||
}; | ||
|
||
MatrixClient.prototype._makeKeyBackupPath = function(roomId, sessionId, version) { | ||
let path; | ||
if (sessionId !== undefined) { | ||
path = utils.encodeUri("/room_keys/keys/$roomId/$sessionId", { | ||
$roomId: roomId, | ||
$sessionId: sessionId, | ||
}); | ||
} else if (roomId !== undefined) { | ||
path = utils.encodeUri("/room_keys/keys/$roomId", { | ||
$roomId: roomId, | ||
}); | ||
} else { | ||
path = "/room_keys/keys"; | ||
} | ||
const queryData = version === undefined ? undefined : { version: version }; | ||
return { | ||
path: path, | ||
queryData: queryData, | ||
}; | ||
}; | ||
|
||
/** | ||
* Back up session keys to the homeserver. | ||
* @param {string} roomId ID of the room that the keys are for Optional. | ||
* @param {string} sessionId ID of the session that the keys are for Optional. | ||
* @param {integer} version backup version Optional. | ||
* @param {object} data Object keys to send | ||
* @return {module:client.Promise} a promise that will resolve when the keys | ||
* are uploaded | ||
*/ | ||
MatrixClient.prototype.sendKeyBackup = function(roomId, sessionId, version, data) { | ||
if (this._crypto === null) { | ||
throw new Error("End-to-end encryption disabled"); | ||
} | ||
|
||
const path = this._makeKeyBackupPath(roomId, sessionId, version); | ||
return this._http.authedRequest( | ||
undefined, "PUT", path.path, path.queryData, data, | ||
); | ||
}; | ||
|
||
MatrixClient.prototype.backupAllGroupSessions = function(version) { | ||
if (this._crypto === null) { | ||
throw new Error("End-to-end encryption disabled"); | ||
} | ||
|
||
return this._crypto.backupAllGroupSessions(version); | ||
}; | ||
|
||
MatrixClient.prototype.isValidRecoveryKey = function(recoveryKey) { | ||
try { | ||
decodeRecoveryKey(recoveryKey); | ||
return true; | ||
} catch (e) { | ||
return false; | ||
} | ||
}; | ||
|
||
MatrixClient.prototype.restoreKeyBackups = function( | ||
recoveryKey, targetRoomId, targetSessionId, version, | ||
) { | ||
if (this._crypto === null) { | ||
throw new Error("End-to-end encryption disabled"); | ||
} | ||
let totalKeyCount = 0; | ||
let keys = []; | ||
|
||
const path = this._makeKeyBackupPath(targetRoomId, targetSessionId, version); | ||
|
||
// FIXME: see the FIXME in createKeyBackupVersion | ||
const privkey = decodeRecoveryKey(recoveryKey); | ||
const decryption = new global.Olm.PkDecryption(); | ||
try { | ||
decryption.init_with_private_key(privkey); | ||
} catch(e) { | ||
decryption.free(); | ||
throw e; | ||
} | ||
|
||
return this._http.authedRequest( | ||
undefined, "GET", path.path, path.queryData, | ||
).then((res) => { | ||
if (res.rooms) { | ||
for (const [roomId, roomData] of Object.entries(res.rooms)) { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ooh, i had no idea you could do that |
||
if (!roomData.sessions) continue; | ||
|
||
totalKeyCount += Object.keys(roomData.sessions).length; | ||
const roomKeys = keysFromRecoverySession( | ||
roomData.sessions, decryption, roomId, roomKeys, | ||
); | ||
for (const k of roomKeys) { | ||
k.room_id = roomId; | ||
keys.push(k); | ||
} | ||
} | ||
} else if (res.sessions) { | ||
totalKeyCount = Object.keys(res.sessions).length; | ||
keys = keysFromRecoverySession( | ||
res.sessions, decryption, targetRoomId, keys, | ||
); | ||
} else { | ||
totalKeyCount = 1; | ||
try { | ||
const key = keyFromRecoverySession(res, decryption); | ||
key.room_id = targetRoomId; | ||
key.session_id = targetSessionId; | ||
keys.push(key); | ||
} catch (e) { | ||
console.log("Failed to decrypt session from backup"); | ||
} | ||
} | ||
|
||
return this.importRoomKeys(keys); | ||
}).then(() => { | ||
return {total: totalKeyCount, imported: keys.length}; | ||
}).finally(() => { | ||
decryption.free(); | ||
}); | ||
}; | ||
|
||
MatrixClient.prototype.deleteKeysFromBackup = function(roomId, sessionId, version) { | ||
if (this._crypto === null) { | ||
throw new Error("End-to-end encryption disabled"); | ||
} | ||
|
||
const path = this._makeKeyBackupPath(roomId, sessionId, version); | ||
return this._http.authedRequest( | ||
undefined, "DELETE", path.path, path.queryData, | ||
); | ||
}; | ||
|
||
// Group ops | ||
// ========= | ||
// Operations on groups that come down the sync stream (ie. ones the | ||
|
@@ -3735,6 +4063,24 @@ module.exports.CRYPTO_ENABLED = CRYPTO_ENABLED; | |
* }); | ||
*/ | ||
|
||
/** | ||
* Fires whenever the status of e2e key backup changes, as returned by getKeyBackupEnabled() | ||
* @event module:client~MatrixClient#"crypto.keyBackupStatus" | ||
* @param {bool} enabled true if key backup has been enabled, otherwise false | ||
* @example | ||
* matrixClient.on("crypto.keyBackupStatus", function(enabled){ | ||
* if (enabled) { | ||
* [...] | ||
* } | ||
* }); | ||
*/ | ||
|
||
/** | ||
* Fires when we want to suggest to the user that they restore their megolm keys | ||
* from backup or by cross-signing the device. | ||
* | ||
* @event module:client~MatrixClient#"crypto.suggestKeyRestore" | ||
*/ | ||
|
||
// EventEmitter JSDocs | ||
|
||
|
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This FIXME seems to be obsolete, since I don't see any relevant FIXME in createKeyBackupVersion. (The one FIXME in there seems unrelated.)