Mercurial > hg > silvermirror
comparison silvermirror-whitepaper.txt @ 14:5f95af14b51c
notes
author | Jeff Hammel <jhammel@mozilla.com> |
---|---|
date | Sun, 24 Mar 2013 12:46:35 -0700 |
parents | 584a847d2491 |
children |
comparison
equal
deleted
inserted
replaced
13:584a847d2491 | 14:5f95af14b51c |
---|---|
1 SilverMirror Whitepaper | 1 = SilverMirror Whitepaper = |
2 | 2 |
3 It is necessary to maintain parallel directory structures of various | 3 It is necessary to maintain parallel directory structures of various |
4 resources across an arbitrary number of computers. The traditional | 4 resources across an arbitrary number of computers. The traditional |
5 approach is the central server model, where files live in one | 5 approach is the central server model, where files live in one |
6 canonical location and the network is used to give access to the data. | 6 canonical location and the network is used to give access to the data. |
10 | 10 |
11 Distributed version control, often of nominal use (in the case where a | 11 Distributed version control, often of nominal use (in the case where a |
12 canonical trunk exists) is ideally suited to provide mirroring of | 12 canonical trunk exists) is ideally suited to provide mirroring of |
13 desired resources across computers. | 13 desired resources across computers. |
14 | 14 |
15 Implementation | 15 == Implementation == |
16 | 16 |
17 A front end to a DVCS - most likely mercurial but potentially bzr - | 17 A front end to a DVCS or other - unison is completed, |
18 mercurial is up next - | |
18 will be written to keep resources in sync across an arbitrary number | 19 will be written to keep resources in sync across an arbitrary number |
19 of computers. The front end, called SilverMirror, may be used to push | 20 of computers. The front end, called SilverMirror, may be used to push |
20 or pull changes to resources. Optionally, a daemon will monitor | 21 or pull changes to resources. Optionally, a daemon will monitor |
21 changes to resources and push or pull changes at desired intervals. | 22 changes to resources and push or pull changes at desired intervals. |
22 | 23 |
26 versioned, any change to the resource's directory structure should be | 27 versioned, any change to the resource's directory structure should be |
27 mirrored across machines without user intervention. Files matching a | 28 mirrored across machines without user intervention. Files matching a |
28 pattern may be ignored, either globally or on a per resource basis, | 29 pattern may be ignored, either globally or on a per resource basis, |
29 for the purpose of versioning. | 30 for the purpose of versioning. |
30 | 31 |
31 Configuration | 32 == Configuration == |
32 | 33 |
33 SilverMirror is configured via an INI file containing a section for | 34 SilverMirror is configured via an INI file containing a section for |
34 each resource and a section for application configuration. | 35 each resource and a section for application configuration. |
35 | 36 |
36 The main section, denoted [::SilverMirror::], has the following options: | 37 The main section, denoted [::SilverMirror::], has the following options: |
59 * hosts: hosts to push/pull from | 60 * hosts: hosts to push/pull from |
60 | 61 |
61 In order to ensure coherency among resources, all relevant | 62 In order to ensure coherency among resources, all relevant |
62 configuration options must be synced prior to push/pull transactions. | 63 configuration options must be synced prior to push/pull transactions. |
63 | 64 |
64 Default Configuration: | 65 Default Configuration:: |
65 | 66 |
66 [::SilverMirror::] | 67 [::SilverMirror::] |
67 conflict = ClobberRemote | 68 conflict = ClobberRemote |
68 | 69 |
69 Example of a more complex configuration: | 70 Example of a more complex configuration:: |
70 | 71 |
71 [::SilverMirror::] | 72 [::SilverMirror::] |
72 conflict.push = ClobberRemote | 73 conflict.push = ClobberRemote |
73 conflict.pull = ClobberLocal | 74 conflict.pull = ClobberLocal |
74 | 75 |
75 Push | 76 == Push == |
76 | 77 |
77 Push changes to remote resources. When resources are pushed, first | 78 Push changes to remote resources. When resources are pushed, first |
78 changes are pulled from each remote host in turn, conflicts between | 79 changes are pulled from each remote host in turn, conflicts between |
79 local and remote changes are resolved (see Behavior on Conflicts), | 80 local and remote changes are resolved (see Behavior on Conflicts), |
80 then local modifications are pushed. This is done to keep the | 81 then local modifications are pushed. This is done to keep the |
83 When new files are added to the resource they are automatically added | 84 When new files are added to the resource they are automatically added |
84 to the hg repository. When resource files are edited the changes are | 85 to the hg repository. When resource files are edited the changes are |
85 pushed to the repository. When a conflict occurs between local | 86 pushed to the repository. When a conflict occurs between local |
86 resources and remote resources, the conflict handler is used. | 87 resources and remote resources, the conflict handler is used. |
87 | 88 |
88 Pull | 89 == Pull == |
89 | 90 |
90 Get changes to the cloud filesystem resources. If no host is | 91 Get changes to the cloud filesystem resources. If no host is |
91 specified, pull changes from all known + accessible hosts. | 92 specified, pull changes from all known + accessible hosts. |
92 | 93 |
93 Namespaced Resources | 94 == Namespaced Resources == |
94 | 95 |
95 It is possible to maintain versioning of a subdirectory within a | 96 It is possible to maintain versioning of a subdirectory within a |
96 resource. | 97 resource. |
97 | 98 |
98 Example: | 99 Example:: |
99 | 100 |
100 [docs] | 101 [docs] |
101 directory = /path/to/docs | 102 directory = /path/to/docs |
102 | 103 |
103 [docs:private] | 104 [docs:private] |
104 | 105 |
105 This configuration snippet describes a resource, [docs:private], | 106 This configuration snippet describes a resource, [docs:private], |
106 namespaced within the [docs] resource. [docs:private] inherits | 107 namespaced within the [docs] resource. [docs:private] inherits |
107 configuration and behavior from [docs] but may be dealt with | 108 configuration and behavior from [docs] but may be dealt with |
108 separately. For example, some computers in the cloud may not have | 109 separately. For example, some computers in the cloud may not have |
118 from its namespace (private) and the directory of its parent resource. | 119 from its namespace (private) and the directory of its parent resource. |
119 So its base directory is /path/to/docs/private . If a relative path | 120 So its base directory is /path/to/docs/private . If a relative path |
120 was specified in the directory option of the [docs:private] section, | 121 was specified in the directory option of the [docs:private] section, |
121 it would be joined with the base directory of [docs]. | 122 it would be joined with the base directory of [docs]. |
122 | 123 |
123 Behavior on Conflicts | 124 == Behavior on Conflicts == |
124 | 125 |
125 Conflict handlers are set via setuptools entry points. Several | 126 Conflict handlers are set via setuptools entry points. Several |
126 conflict handlers are provided with SilverMirror: | 127 conflict handlers are provided with SilverMirror: |
127 | 128 |
128 * ClobberLocal: replace local changes with changes from remote files | 129 * ClobberLocal: replace local changes with changes from remote files |
135 | 136 |
136 The conflict handler may also be specified from the command line: | 137 The conflict handler may also be specified from the command line: |
137 | 138 |
138 silvermirror push -d ClobberRemote | 139 silvermirror push -d ClobberRemote |
139 | 140 |
140 Command Line Usage | 141 == Command Line Usage == |
141 | 142 |
142 silvermirror [push|pull] [resource] [options] | 143 silvermirror [push|pull] [resource] [options] |
143 | 144 |
144 In the simplest invocation, SilverMirror is used with no command line | 145 In the simplest invocation, SilverMirror is used with no command line |
145 arguments: | 146 arguments:: |
146 | 147 |
147 silvermirror | 148 silvermirror |
148 | 149 |
149 This pushes changes of the resource as determined by the current | 150 This pushes changes of the resource as determined by the current |
150 working directory after pulling outstanding changes from all | 151 working directory after pulling outstanding changes from all |
151 applicable remote computers and invoking the conflict handler for | 152 applicable remote computers and invoking the conflict handler for |
152 push. If the current working directory is not within a resource, all | 153 push. If the current working directory is not within a resource, all |
189 SilverMirror does not desire to duplicate versioning on directories | 190 SilverMirror does not desire to duplicate versioning on directories |
190 already under version control (svn, bzr, hg). These resources are | 191 already under version control (svn, bzr, hg). These resources are |
191 automatically ignore. In a future implementation, these resources | 192 automatically ignore. In a future implementation, these resources |
192 would optionally be checked out or updated upon a pull. | 193 would optionally be checked out or updated upon a pull. |
193 | 194 |
194 Automatic Syncronization | 195 == Automatic Syncronization == |
195 | 196 |
196 SilverMirror includes a script that will automatically invoke | 197 SilverMirror includes a script that will automatically invoke |
197 syncronizing the resources in a specified period of time. This daemon, | 198 syncronizing the resources in a specified period of time. This daemon, |
198 called silvermirrord, is invoked from the command line with options | 199 called silvermirrord, is invoked from the command line with options |
199 parallel to the silvermirror program. One additional option, -s, tells | 200 parallel to the silvermirror program. One additional option, -s, tells |
203 is noninteractive, otherwise the daemon will hang forever. | 204 is noninteractive, otherwise the daemon will hang forever. |
204 | 205 |
205 As an alternative, the silvermirror program may be invoked from a cron | 206 As an alternative, the silvermirror program may be invoked from a cron |
206 job. | 207 job. |
207 | 208 |
208 Future Work | 209 == Events == |
210 | |
211 SilverMirror [TODO] may be hooked up to an event listener for | |
212 filesystem changes and sync on them. | |
213 | |
214 == Future Work == | |
209 | 215 |
210 SilverMirror implements a cloud filesystem which may be accessed | 216 SilverMirror implements a cloud filesystem which may be accessed |
211 nearly transparently by an arbitrary number of computers. Several | 217 nearly transparently by an arbitrary number of computers. Several |
212 improvements could extend SilverMirror to solve several deficiencies | 218 improvements could extend SilverMirror to solve several deficiencies |
213 of modern filesystem. | 219 of modern filesystem. |
233 SilverMirror backup scheme on portable media. This not only | 239 SilverMirror backup scheme on portable media. This not only |
234 includes a backup of the files, but also the SilverMirror program and | 240 includes a backup of the files, but also the SilverMirror program and |
235 all of the files necessary to create a new SilverMirror node. | 241 all of the files necessary to create a new SilverMirror node. |
236 | 242 |
237 * SilverMirror + Firefox Sync | 243 * SilverMirror + Firefox Sync |
244 | |
245 == Vision == | |
246 | |
247 Essentially, SilverMirror is intended as a portable filesystem on a | |
248 filesystem type interface. |