gdritter repos melvil / 87eae2c
Started setting up Python packaging properly Getty Ritter 8 years ago
7 changed file(s) with 367 addition(s) and 326 deletion(s). Collapse all Expand all
+92
-0
README less more
1 # aloysius
2
3 **EARLY AND EXPERIMENTAL**
4
5 Aloysius is the HTTP server interface I want to use. It's very
6 slow at present, and still quite early, but it's at least a
7 proof-of-concept of something I think should exist.
8
9 ## Basic Use
10
11 The Aloysius server does nothing but pass HTTP requests and
12 responses between other servers: it is, in effect, a mechanism
13 for establishing reverse proxies.
14
15 The server is invoked with a single optional argument, and it
16 continues running in the foreground until it is killed with
17 standard Unix signals. The argument is a directory, and if that
18 directory exists, it switches to that directory before
19 continuing. It then reads configuration from that directory and
20 will continuously forward requests based on that configuration.
21
22 The configuration directory contains zero or more
23 subdirectories, each of which describes a given request filter
24 and forwarding mechanism. The subdirectory may contain several
25 specifically named files, whose contents specify a forwarding
26 system:
27
28 ~~~
29 path: which request paths to match; defaults to "*"
30 domain: which request subdomains to match; defaults to "*"
31 mode: how to forward the request; defaults to "http"
32 host: which host to forward to; defaults to "localhost"
33 port: which port to forward to; defaults to "80"
34 conf: which path to forward to; defaults to "/dev/null"
35 resp: which HTTP response to issue; defaults to 303
36 ~~~
37
38 These are interpreted as follows:
39
40 - The `path` and `domain` fields tell us which requests to forward:
41 both of them default to accepting anything, and both of them
42 allow their values to have the wildcard character `*`.
43
44 - The `mode` field tells us _how_ to forward requests. There are
45 three possible forwarding modes:
46 - If the mode is `http`, then Aloysius will forward the HTTP
47 request to the server listening on the host `host` and the
48 port `port`.
49 - If the mode is `aloys`, then Aloysius will recursively check
50 the configuration directory at `conf`.
51 - If the mode is `redir`, then Aloysius will respond with an
52 HTTP response code as indicated in `resp` and redirect to
53 the host as indicated in `host`.
54
55 ## Example Setups
56
57 Because configuration is specified as a directory, rather than as
58 a single file, we can use properties of the Unix file system as a
59 simple ACL-like mechanism. For example, a system administrator
60 can set up a user-owned configuration directory for each user,
61 and then use a global configuration directory to forward requests
62 to that user on a per-subdomain basis:
63
64 ~~~
65 $ mkdir -p /var/run/aloys
66 $ for U in $USERS
67 > do
68 > # find the user's home directory
69 > HOMEDIR=`cat /etc/passwd | grep ${U} | cut -d ':' -f 6`
70 >
71 > # add a configuration directory to each user
72 > mkdir -p ${HOMEDIR}/aloys
73 > chown ${U} ${HOMEDIR}/aloys
74 >
75 > # add a new forwarding rule for each user
76 > mkdir -p /var/run/aloys/${U}-local
77 > # make ${U}.example.com forward to the user's aloys configuration
78 > echo "${U}.example.com" >/var/run/aloys/user-${U}/domain
79 > echo "aloys" >/var/run/aloys/user-${U}/mode
80 > echo "${HOMEDIR}/aloys" >/var/run/aloys/user-${U}/conf
81 > done
82 $ aloysius /var/run/aloys
83 ~~~
84
85 Now, if a given user wants to set up a local HTTP server that
86 produces dynamic content, they can add the appropriate forwarding
87 configuration to their own directory, but they cannot modify
88 other users' configurations or the global configuration.
89
90 Even if you're running a single server, but want to have multiple
91 services on it, this can be a convenient way to set up reverse
92 proxy servers without needing root access.
+0
-92
README.md less more
1 # aloysius
2
3 **EARLY AND EXPERIMENTAL**
4
5 Aloysius is the HTTP server interface I want to use. It's very
6 slow at present, and still quite early, but it's at least a
7 proof-of-concept of something I think should exist.
8
9 ## Basic Use
10
11 The Aloysius server does nothing but pass HTTP requests and
12 responses between other servers: it is, in effect, a mechanism
13 for establishing reverse proxies.
14
15 The server is invoked with a single optional argument, and it
16 continues running in the foreground until it is killed with
17 standard Unix signals. The argument is a directory, and if that
18 directory exists, it switches to that directory before
19 continuing. It then reads configuration from that directory and
20 will continuously forward requests based on that configuration.
21
22 The configuration directory contains zero or more
23 subdirectories, each of which describes a given request filter
24 and forwarding mechanism. The subdirectory may contain several
25 specifically named files, whose contents specify a forwarding
26 system:
27
28 ~~~
29 path: which request paths to match; defaults to "*"
30 domain: which request subdomains to match; defaults to "*"
31 mode: how to forward the request; defaults to "http"
32 host: which host to forward to; defaults to "localhost"
33 port: which port to forward to; defaults to "80"
34 conf: which path to forward to; defaults to "/dev/null"
35 resp: which HTTP response to issue; defaults to 303
36 ~~~
37
38 These are interpreted as follows:
39
40 - The `path` and `domain` fields tell us which requests to forward:
41 both of them default to accepting anything, and both of them
42 allow their values to have the wildcard character `*`.
43
44 - The `mode` field tells us _how_ to forward requests. There are
45 three possible forwarding modes:
46 - If the mode is `http`, then Aloysius will forward the HTTP
47 request to the server listening on the host `host` and the
48 port `port`.
49 - If the mode is `aloys`, then Aloysius will recursively check
50 the configuration directory at `conf`.
51 - If the mode is `redir`, then Aloysius will respond with an
52 HTTP response code as indicated in `resp` and redirect to
53 the host as indicated in `host`.
54
55 ## Example Setups
56
57 Because configuration is specified as a directory, rather than as
58 a single file, we can use properties of the Unix file system as a
59 simple ACL-like mechanism. For example, a system administrator
60 can set up a user-owned configuration directory for each user,
61 and then use a global configuration directory to forward requests
62 to that user on a per-subdomain basis:
63
64 ~~~
65 $ mkdir -p /var/run/aloys
66 $ for U in $USERS
67 > do
68 > # find the user's home directory
69 > HOMEDIR=`cat /etc/passwd | grep ${U} | cut -d ':' -f 6`
70 >
71 > # add a configuration directory to each user
72 > mkdir -p ${HOMEDIR}/aloys
73 > chown ${U} ${HOMEDIR}/aloys
74 >
75 > # add a new forwarding rule for each user
76 > mkdir -p /var/run/aloys/${U}-local
77 > # make ${U}.example.com forward to the user's aloys configuration
78 > echo "${U}.example.com" >/var/run/aloys/user-${U}/domain
79 > echo "aloys" >/var/run/aloys/user-${U}/mode
80 > echo "${HOMEDIR}/aloys" >/var/run/aloys/user-${U}/conf
81 > done
82 $ aloysius /var/run/aloys
83 ~~~
84
85 Now, if a given user wants to set up a local HTTP server that
86 produces dynamic content, they can add the appropriate forwarding
87 configuration to their own directory, but they cannot modify
88 other users' configurations or the global configuration.
89
90 Even if you're running a single server, but want to have multiple
91 services on it, this can be a convenient way to set up reverse
92 proxy servers without needing root access.
+38
-0
aloysius/aloysius.py less more
1 from twisted.web.client import Agent
2 from twisted.web.server import Site
3 from twisted.web.resource import Resource
4 from twisted.internet import reactor
5 import os
6 import sys
7 import yaml
8 import routeconf
9
10
11 class Sample(Resource):
12 isLeaf = True
13
14 def __init__(self, routes, agent):
15 Resource.__init__(self)
16 self.routes = routes
17 self.agent = agent
18
19 def render_GET(self, request):
20 routeconf.domain_for(request)
21 return self.routes.handle(request, self.agent)
22
23
24 def main():
25 if len(sys.argv) > 1:
26 conf_dir = sys.argv[1]
27 else:
28 conf_dir = os.getenv('ALOYS_DIR', os.getcwd())
29 port = int(os.getenv('PORT', 8080))
30
31 routes = routeconf.load_routes(conf_dir)
32 agent = Agent(reactor)
33 sys.stderr.write(routes.pretty())
34 reactor.listenTCP(port, Site(Sample(routes, agent)))
35 reactor.run()
36
37 if __name__ == '__main__':
38 main()
+208
-0
aloysius/routeconf.py less more
1 import os
2 import sys
3 from twisted.web.server import NOT_DONE_YET
4 from twisted.web.client import readBody
5
6
7 def load_routes(location):
8 routes = []
9 if not os.path.isdir(location):
10 raise Exception(
11 'Cannot find specified configure directory: {0}', location)
12 for site in sorted(os.listdir(location)):
13 try:
14 routes.append(load_site(os.path.join(location, site)))
15 except Exception as e:
16 sys.stderr.write('Unable to load {0}; skipping\n'.format(
17 os.path.abspath(os.path.join(location, site))))
18 sys.stderr.write(' {0}\n\n'.format(e))
19 return RouteList(routes)
20
21
22 def try_load(filename, default=None):
23 if not os.path.exists(filename):
24 return default
25 if not os.path.isfile(os.path.abspath(filename)):
26 raise Exception(
27 'Cannot load {0}: is not a file'.format(filename))
28 with open(filename) as f:
29 return f.read().strip()
30
31
32 def domain_for(request):
33 return request.getHeader('host')
34
35
36 def load_site(loc):
37 if not os.path.isdir(loc):
38 raise Exception('Site spec not a directory: {0}'.format(loc))
39
40 if (not os.path.exists(os.path.join(loc, 'domain')) and
41 not os.path.exists(os.path.join(loc, 'path'))):
42 raise Exception(
43 'The site {0} does not specify a domain or a path'.format(loc))
44 domain = try_load(os.path.join(loc, 'domain'), '*')
45 path = try_load(os.path.join(loc, 'path'), '*').lstrip('/')
46 mode = try_load(os.path.join(loc, 'mode'), 'http')
47 if mode == 'http':
48 dispatch = HTTPDispatch(
49 host=try_load(os.path.join(loc, 'host'), 'localhost'),
50 port=int(try_load(os.path.join(loc, 'port'), 80)))
51 elif mode == 'aloys':
52 dispatch = AloysDispatch(
53 loc=try_load(os.path.join(loc, 'conf'), '/dev/null'))
54 elif mode == 'redir':
55 dispatch = RedirDispatch(
56 resp=int(try_load(os.path.join(loc, 'resp'), 303)),
57 location=try_load(os.path.join(loc, 'location'),
58 'http://localhost/'))
59 else:
60 raise Exception('Unknown mode: {0}'.format(repr(mode)))
61 return Route(domain=domain, path=path, dispatch=dispatch, disk_loc=loc)
62
63
64 def wildcard_match(spec, string):
65 loc_stack = [(0, 0)]
66 while loc_stack:
67 (i, j) = loc_stack.pop()
68 if i >= len(spec) and j >= len(string):
69 return True
70 elif i >= len(spec) or j >= len(string):
71 continue
72 elif spec[i] == string[j]:
73 loc_stack.append((i + 1, j + 1))
74 elif spec[i] == '*':
75 loc_stack.append((i + 1, j + 1))
76 loc_stack.append((i, j + 1))
77 else:
78 continue
79 return False
80
81
82 class RouteList:
83 '''
84 An object which represents an ordered set of possible routing
85 options.
86 '''
87
88 def __init__(self, routes):
89 self.routes = routes
90
91 def handle(self, request, agent):
92 for route in self.routes:
93 if route.matches(request):
94 return route.handle(request, agent)
95 return "unable to handle request"
96
97 def __repr__(self):
98 return 'RouteList([{0}])'.format(
99 ', '.join(repr(r) for r in self.routes))
100
101 def pretty(self, level=0):
102 return ''.join(
103 ' ' * level + r.pretty(level=level)
104 for r in self.routes)
105
106
107 class Route:
108
109 def __init__(self, domain, path, dispatch, disk_loc=''):
110 self.domain = domain
111 self.path = path
112 self.dispatch = dispatch
113 self.disk_loc = disk_loc
114
115 def matches(self, request):
116 return (wildcard_match(self.path, '/' + request.path) and
117 wildcard_match(self.domain, domain_for(request)))
118
119 def handle(self, request, agent):
120 print 'handled by {0}'.format(self.disk_loc)
121 return self.dispatch.handle(request, agent)
122
123 def __repr__(self):
124 return 'Route({0}, {1}, {2}, disk_loc={3})'.format(
125 repr(self.domain),
126 repr(self.path),
127 self.dispatch,
128 repr(self.disk_loc))
129
130 def pretty(self, level=0):
131 return '{0}/{1} => {2}'.format(self.domain,
132 self.path,
133 self.dispatch.pretty(level=level))
134
135
136 class Dispatch:
137
138 def handle(self, request, agent):
139 return '[no route matched your request]'
140
141 def __repr__(self):
142 return '{name}({args})'.format(
143 name=self.__class__.__name__,
144 args=', '.join('{0}={1}'.format(k, repr(v))
145 for (k, v) in self.__dict__.items()))
146
147
148 class HTTPDispatch(Dispatch):
149
150 def __init__(self, host='localhost', port=80):
151 self.host = host
152 self.port = port
153
154 def handle(self, request, agent):
155 def callback(response):
156 for (k, v) in response.headers.getAllRawHeaders():
157 request.setHeader(k, v[0])
158 r = readBody(response)
159 r.addCallback(handle_body)
160 return r
161
162 def handle_body(body):
163 request.write(body)
164 request.finish()
165
166 url = 'http://{0}:{1}{2}'.format(
167 self.host, self.port, request.path)
168 print url
169 agent.request('GET', url, request.requestHeaders, None) \
170 .addCallback(callback)
171 return NOT_DONE_YET
172
173 def pretty(self, level=0):
174 return '{0}:{1}\n'.format(self.host, self.port)
175
176
177 class AloysDispatch(Dispatch):
178 '''
179 Dispatch the route down to a different configuration directory.
180 '''
181
182 def __init__(self, loc):
183 self.routes = load_routes(loc)
184
185 def handle(self, request, agent):
186 return self.routes.handle(request)
187
188 def pretty(self, level=0):
189 return '\n' + self.routes.pretty(level=level + 2)
190
191
192 class RedirDispatch(Dispatch):
193 '''
194 Use a redirect code (probably 303) to manually redirect to a
195 different location.
196 '''
197
198 def __init__(self, resp=303, location='http://localhost/'):
199 self.resp = resp
200 self.location = location
201
202 def handle(self, request, agent):
203 request.setResponseCode(self.resp)
204 request.setHeader('location', self.location)
205 return ''
206
207 def pretty(self, level=0):
208 return 'redir({0})\n'.format(self.location)
+29
-0
setup.py less more
1 import os
2 from setuptools import find_packages, setup
3
4
5 def read_file(filename):
6 '''
7 Read the contents of a file relative to this one.
8 '''
9 with open(os.path.join(os.path.dirname(__file__), filename)) as f:
10 return f.read()
11
12 setup(
13 name='aloysius',
14 version='0.0.6',
15 author='Getty Ritter',
16 author_email='getty.ritter@gmail.com',
17 description='A simple, configurable HTTP reverse proxy.',
18 install_requires='twisted',
19 packages=find_packages(),
20 license='BSD',
21 keywords='http',
22 url='http://infinitenegativeutility.com/software/aloysius',
23 long_description=read_file('README'),
24 entry_points={
25 'console_scripts': [
26 'aloysius = aloysius:main'
27 ]
28 },
29 )
+0
-26
src/aloysius.py less more
1 from twisted.web.client import Agent
2 from twisted.web.server import Site
3 from twisted.web.resource import Resource
4 from twisted.internet import reactor
5 import yaml
6 import routeconf
7
8
9 class Sample(Resource):
10 isLeaf = True
11
12 def __init__(self, routes, agent):
13 Resource.__init__(self)
14 self.routes = routes
15 self.agent = agent
16
17 def render_GET(self, request):
18 routeconf.domain_for(request)
19 return self.routes.handle(request, self.agent)
20
21 if __name__ == '__main__':
22 routes = routeconf.load_routes('sample')
23 agent = Agent(reactor)
24 print routes.pretty()
25 reactor.listenTCP(8080, Site(Sample(routes, agent)))
26 reactor.run()
+0
-208
src/routeconf.py less more
1 import os
2 import sys
3 from twisted.web.server import NOT_DONE_YET
4 from twisted.web.client import readBody
5
6
7 def load_routes(location):
8 routes = []
9 if not os.path.isdir(location):
10 raise Exception(
11 'Cannot find specified configure directory: {0}', location)
12 for site in sorted(os.listdir(location)):
13 try:
14 routes.append(load_site(os.path.join(location, site)))
15 except Exception as e:
16 sys.stderr.write('Unable to load {0}; skipping\n'.format(
17 os.path.abspath(os.path.join(location, site))))
18 sys.stderr.write(' {0}\n\n'.format(e))
19 return RouteList(routes)
20
21
22 def try_load(filename, default=None):
23 if not os.path.exists(filename):
24 return default
25 if not os.path.isfile(os.path.abspath(filename)):
26 raise Exception(
27 'Cannot load {0}: is not a file'.format(filename))
28 with open(filename) as f:
29 return f.read().strip()
30
31
32 def domain_for(request):
33 return request.getHeader('host')
34
35
36 def load_site(loc):
37 if not os.path.isdir(loc):
38 raise Exception('Site spec not a directory: {0}'.format(loc))
39
40 if (not os.path.exists(os.path.join(loc, 'domain')) and
41 not os.path.exists(os.path.join(loc, 'path'))):
42 raise Exception(
43 'The site {0} does not specify a domain or a path'.format(loc))
44 domain = try_load(os.path.join(loc, 'domain'), '*')
45 path = try_load(os.path.join(loc, 'path'), '*').lstrip('/')
46 mode = try_load(os.path.join(loc, 'mode'), 'http')
47 if mode == 'http':
48 dispatch = HTTPDispatch(
49 host=try_load(os.path.join(loc, 'host'), 'localhost'),
50 port=int(try_load(os.path.join(loc, 'port'), 80)))
51 elif mode == 'aloys':
52 dispatch = AloysDispatch(
53 loc=try_load(os.path.join(loc, 'conf'), '/dev/null'))
54 elif mode == 'redir':
55 dispatch = RedirDispatch(
56 resp=int(try_load(os.path.join(loc, 'resp'), 303)),
57 location=try_load(os.path.join(loc, 'location'),
58 'http://localhost/'))
59 else:
60 raise Exception('Unknown mode: {0}'.format(repr(mode)))
61 return Route(domain=domain, path=path, dispatch=dispatch, disk_loc=loc)
62
63
64 def wildcard_match(spec, string):
65 loc_stack = [(0, 0)]
66 while loc_stack:
67 (i, j) = loc_stack.pop()
68 if i >= len(spec) and j >= len(string):
69 return True
70 elif i >= len(spec) or j >= len(string):
71 continue
72 elif spec[i] == string[j]:
73 loc_stack.append((i + 1, j + 1))
74 elif spec[i] == '*':
75 loc_stack.append((i + 1, j + 1))
76 loc_stack.append((i, j + 1))
77 else:
78 continue
79 return False
80
81
82 class RouteList:
83 '''
84 An object which represents an ordered set of possible routing
85 options.
86 '''
87
88 def __init__(self, routes):
89 self.routes = routes
90
91 def handle(self, request, agent):
92 for route in self.routes:
93 if route.matches(request):
94 return route.handle(request, agent)
95 return "unable to handle request"
96
97 def __repr__(self):
98 return 'RouteList([{0}])'.format(
99 ', '.join(repr(r) for r in self.routes))
100
101 def pretty(self, level=0):
102 return ''.join(
103 ' ' * level + r.pretty(level=level)
104 for r in self.routes)
105
106
107 class Route:
108
109 def __init__(self, domain, path, dispatch, disk_loc=''):
110 self.domain = domain
111 self.path = path
112 self.dispatch = dispatch
113 self.disk_loc = disk_loc
114
115 def matches(self, request):
116 return (wildcard_match(self.path, '/' + request.path) and
117 wildcard_match(self.domain, domain_for(request)))
118
119 def handle(self, request, agent):
120 print 'handled by {0}'.format(self.disk_loc)
121 return self.dispatch.handle(request, agent)
122
123 def __repr__(self):
124 return 'Route({0}, {1}, {2}, disk_loc={3})'.format(
125 repr(self.domain),
126 repr(self.path),
127 self.dispatch,
128 repr(self.disk_loc))
129
130 def pretty(self, level=0):
131 return '{0}/{1} => {2}'.format(self.domain,
132 self.path,
133 self.dispatch.pretty(level=level))
134
135
136 class Dispatch:
137
138 def handle(self, request, agent):
139 return '[no route matched your request]'
140
141 def __repr__(self):
142 return '{name}({args})'.format(
143 name=self.__class__.__name__,
144 args=', '.join('{0}={1}'.format(k, repr(v))
145 for (k, v) in self.__dict__.items()))
146
147
148 class HTTPDispatch(Dispatch):
149
150 def __init__(self, host='localhost', port=80):
151 self.host = host
152 self.port = port
153
154 def handle(self, request, agent):
155 def callback(response):
156 for (k, v) in response.headers.getAllRawHeaders():
157 request.setHeader(k, v[0])
158 r = readBody(response)
159 r.addCallback(handle_body)
160 return r
161
162 def handle_body(body):
163 request.write(body)
164 request.finish()
165
166 url = 'http://{0}:{1}{2}'.format(
167 self.host, self.port, request.path)
168 print url
169 agent.request('GET', url, request.requestHeaders, None) \
170 .addCallback(callback)
171 return NOT_DONE_YET
172
173 def pretty(self, level=0):
174 return '{0}:{1}\n'.format(self.host, self.port)
175
176
177 class AloysDispatch(Dispatch):
178 '''
179 Dispatch the route down to a different configuration directory.
180 '''
181
182 def __init__(self, loc):
183 self.routes = load_routes(loc)
184
185 def handle(self, request, agent):
186 return self.routes.handle(request)
187
188 def pretty(self, level=0):
189 return '\n' + self.routes.pretty(level=level + 2)
190
191
192 class RedirDispatch(Dispatch):
193 '''
194 Use a redirect code (probably 303) to manually redirect to a
195 different location.
196 '''
197
198 def __init__(self, resp=303, location='http://localhost/'):
199 self.resp = resp
200 self.location = location
201
202 def handle(self, request, agent):
203 request.setResponseCode(self.resp)
204 request.setHeader('location', self.location)
205 return ''
206
207 def pretty(self, level=0):
208 return 'redir({0})\n'.format(self.location)