Mercurial > hg > toolbox

comparison toolbox-blog.txt @ 2:458f2c48ecce

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
more docs
author Jeff Hammel <k0scist@gmail.com>
date Tue, 19 May 2015 20:08:52 -0700
parents
children
comparison
equal deleted inserted replaced
1:1047e058103d 2:458f2c48ecce
1 Toolbox deployed
2
3 Mozilla has a lot of software tools for a wide class of problems we
4 typically encounter. We're self-motivated people: when we encounter a
5 problem, sometimes we look or ask around to see if anyone else has did
6 anything similar, but if something doesn't turn up (quickly) we'll
7 write it ourselves.
8
9 This is pretty much how (the functional parts of) the Open Source
10 world works. It goes hand in hand with the bazaar model:
11 http://en.wikipedia.org/wiki/The_Cathedral_and_the_Bazaar . The
12 strength of the bazaar model is robustness: the process cannot be
13 stopped by any particular point of failure. You get a robust
14 ecosystem of software that evolves in a manner parallel to biological
15 life.
16
17 The weakness of the bazaar approach, especially when considered as
18 part of an economic system, is that effort is duplicated. The Open
19 Source revolution has made the greatest gains in proliferation: number
20 of languages, number of software projects, and ability to easily fork
21 existing code using distributed version control. But, just as the
22 federated web is harder to achieve than information silos, harvesting
23 and consolidating the vast richness of software is harder than its
24 proliferation and not merely for technical reasons.
25
26 Proliferation was largely a technological problem. This is best
27 illustrated by DVCS. Consider the contrast between forking and
28 merging. With centralized version control, both forking and merging
29 are hard. With DVCS, forking is now trivial. But despite the fact
30 that better tools have been introduced (which actually largely don't
31 have anything to do per se with DVCS), merging is a human problem.
32 Regardless of the amount of cleverness and hours required, technical
33 problems are solvable and will be solved. Human problems can't be
34 solved simply by throwing resources at them.
35
36 Information curation will be the problem of this dawning decade.
37 Using the internet as the predominent informatic presence in peoples'
38 lives, the question has gone from "can this be done?" to "how do I
39 figure out how to do this?" (shadowed only by the overarching
40 question, "why am I doing this?"). Programming has lost its position to
41 UI/UX design.
42
43 Toolbox exists as a prototype to aid in unmixing the software tool
44 soup. It carries two related purposes:
45
46 1. To give people a place to put and find software tools of a wide
47 variety of types, purpose, and sources. To this end, a tool should
48 have a name, a description, a URL, and appropriate metadata. Mozilla
49 tools live in any number of places: in user repos, in shared repos, as
50 web apps, on github, or at any number of off-site hosts. Often, for
51 reasons of convenience and legacy, a useful tool exists packaged [*]
52 as part of another tool or other piece of software. In this sense,
53 toolbox is a software index, similar to PyPI or softpedia.
54
55 2. To give people a tool to curate the existing Mozilla tools towards
56 the end of consolidating similarly purposed code towards discrete
57 high-quality pieces of software. Often, as explained above, code is
58 duplicated. This is done because either an appropriate existing tool
59 could not be found (in an expedient manner) or existing tools were not
60 appropriate to the task at hand. Toolbox, as a software index, should
61 mostly solve the former case given proper participation.
62 If an existing tool is found to be unsuitable for the task at hand,
63 this can be for a few reasons:
64
65 - no such software exists to solve the problem. In this minority case,
66 there is nothing to do but to write the software or reconsider what
67 you're actually trying to do
68
69 - there is software that could be built on, but it is not an expedient
70 solution to the problem. This again can be for several reasons:
71
72 * the modification to the upstream project is easily doable and
73 expedient, but a lengthy review/approval process hampers is
74 practicability to a solution within the time frame desired. This
75 is essentially a problem in the area of deployment/packaging
76 since, in the open source model, the upstream code is freely
77 available, but concerns external to the code itself
78 (i.e. distributability) require more than just the code
79 modifications for continuation of an operable software process [**].
80 In this case, an appropriate methodology is as follows:
81
82 A. The issue is ticketed with the upstream project with the
83 appropriate patch.
84
85 B. A vendor branch is created of the upstream project with the fix
86 applied.
87
88 C. Relevent packaging/deploy systems are updated to point to the
89 vendor branch instead of the upstream project.
90
91 D. Changes to the upstream project are propagated (either manually
92 or automatically) to the vendor branch.
93
94 E. The change is accepted upstream. If there are no other
95 discrepencies between the vendor branch and the origin, the vendor
96 branch is deleted and all references point to the origin. If there
97 are discrepencies, the particular upstreamed change is removed
98 from the vendor branch and fetched from upstream.
99
100 While none of this is hard, it requires time, management
101 (i.e. attention), and adequate tools.
102
103 * the modification won't be accepted upstream. This is similar to
104 the above case, except the vendor branch will never (conceivably)
105 go away. It is usually not possible to distinguish the case up
106 front or in any predictable time-frame. It may also be
107 conceivable to permanantly fork the project; that is, the vendor
108 branch becomes a first-class resource (a project of its own right)
109 versus a temporary necessity that should not (normally) be
110 consumed downstream.
111
112 * The modification to the code is hard in terms of required time to
113 invest, required maintenance, and/or required attention to
114 complexity. In general, the required work and maintenance will be
115 unknown whereas the time to measure from nothing to a working
116 solution may be more estimable.
117
118 - there are one or more softwares that nearly do what you desire, but
119 not quite. This is often due to a misclarity of intent of the
120 software formulators, or software that may have originally had a
121 clear intent but that has been adopted to fulfill more nebulous
122 purposes.
123
124 The above presents a best-practices scenario. In order for open
125 source, as a global village, to sustain and flourish, the strengths of
126 open source -- the unconstrained flow of information, the championship
127 of the individual, cooperative advantage, and community ethics -- must
128 be utilized and best practices must be achieved as the flowerbed is
129 shared with corporate interests to which the open-source ecosystem,
130 while competition for market share mandating utilization via its
131 (often) excellent code base, the proliferation of potentially
132 non-moneyed (and non money-oriented) alternatives represents a direct
133 threat to the power structure[***].
134
135 More directly, effort must be made towards coherency and cohesiveness
136 of open source. How does one bring order to the bazaar?[****] One
137 must begin with an understanding of how the bazaar works. Open source
138 programmers are, by inspection, largely good citizens (after a
139 fashion). They allow consumption and distribution of their work,
140 their source code, without exploit it for a capital
141 advantage[*****]. If the process of organizing is fitted organically
142 into their existing workflow, open source programmers will
143 organize. If the practical advantage to themselves and to the system
144 as a whole of organizing is perceived, a coder will go to the
145 (non-over-burdening) required effort in order to make this so. When
146 first exposed to tools such as issue trackers, VCS, automated testing
147 (e.g.), programmers often first resist adoption of these technologies
148 until a case clearly manifests that the tool in question does (or more
149 often, would have) saved considerable time, effort, and/or
150 status. Then the developer uses the tool and will consider anyone who,
151 as s/he previously did, does not naive. This pointing of arrows is an
152 example of how the bazaar works: individuals are agents and operators of
153 information, but the transfer of information, which is in what makes
154 information valuable, is a transpersonal process. You do not bring
155 order to the bazaar; you plant the seeds and nurture towards a
156 bountiful harvest.
157
158 ------
159
160 [*] using the term loosely
161
162 [**] For instance, consider a software project requiring a one line
163 modification of gcc to build. Assuming even that this one line is a
164 proper and non-contraversial fix for a gcc bug, gcc releases are
165 heavily checked for stability and released only periodically, so the
166 turnaround even given a swift review process will be long and also in
167 general unknowably long. The software project in question can, as a
168 patchwork fix, require its developers to apply the patch to their own
169 gcc source and build their own compiler to continue project
170 development. However, this raises the bar to entry quite a bit,
171 skill-wise, time-wise, and psychologically. In addition, while
172 previously the project could require a particular OOTB version of gcc,
173 following the patch requirement, precise build instructions must be
174 provided and it will become difficult to diagnose what is going wrong
175 with particular compile errors. All legacy developer environments
176 will be rendered non-working and all developers will have to apply the
177 same additional work to get a good development environment going
178 forward unless tools are provided.
179
180 [***] Open source is not unique in this particular position. In fact,
181 the anti-marketing market is a large market. Consider sales of Che
182 Guevera t-shirts, feminist literature from the traditional publishing
183 oligarchy, and Hollywood's marketing of the underdog. In all of these
184 cases, that which is being peddled is -- symbolically and sometimes
185 literally -- in direct philosophic conflict with the marketeer. The
186 market, however, cares only to the degree that this is
187 perceived as an existential threat (which is usually not at all), and
188 subversion may be safely sold and counted only in terms of dollars. If
189 the Recording Industry Association of America ever needed a slush
190 fund, it could easily gain an buffer by selling plain white cotton
191 T-shirts emblazoned with the logo: "Destroy the RIAA".
192
193 [****] The question itself is intentionally flawed
194
195 [*****] There is the case where (a) programmer(s) intentionally
196 (conciouslly or subconciously) will set up a consulting firm
197 installing overly complex house-ware, charging for their expertise,
198 and then running. While this does happen, it is not a major factor in
199 open source at large.