mptcp/wanproxy
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity

added sample configuration

Browse source
This commit is contained in:
Bramfeld Team 2015-09-07 13:24:42 +02:00
parent bde0ea9f70
commit 5ab09e78b2
3 changed files with 57 additions and 270 deletions
Download patch file Download diff file Expand all files Collapse all files

53
LAYOUT
View file

@ -1,53 +0,0 @@
Component directory layout:
Directory Description
============= =============================================================
common Non-specific functionality that is of general interest, e.g.
the logging system, test infrastructure and Buffer class.
common/thread Threading infrastructure.
common/timer Time sampling infrastructure.
common/uuid UUID API.
config General-purpose configuration system infrastructure and some
essential types and classes.
crypto Cryptographic API and implementations, including hashes, MACs
and encryption algorithms.
event The event system and infrastructure to support it, such as
the various implementations of the poll interface.
http HTTP protocol implementation and utility functions.
io IO system, namely file descriptors interfaces.
io/net Networking abstractions for the IO system, utility functions and
protocols.
io/pipe Pipe infrastructure; abstractions and useful types.
io/socket Sockets interface to the IO system.
network Low-level networking, e.g. packet capture.
network/uinet User-space TCP/IP stack (see network/uinet/LAYOUT for futher
layout info).
programs Stand-alone programs, organized into sub-directories named by
program.
programs/diskdup
Block storage demonstration program, for making two block
devices or files have identical contents with a minimal number
of writes.
programs/fwdproxy
Simple file-driven port-forwarding proxy.
programs/tack File-oriented dictionary compression using XCodec.
programs/wanproxy
WANProxy.
programs/websplat
Simple web server.
programs/xcdump XCodec stream decoder and pretty-printer.
ssh SSH protocol implementation.
xcodec The XCodec codec, which performs deduplication, and its
internal interfaces.
xml Trivial XML functions.
zlib Wrappers around zlib, providing Pipes that do inflate and
deflate.
Standard sub-directories:
Directory Description
============= =============================================================
test Unit tests.
example Example programs which may test functionality in a
non-automated fashion or merely provide example usage of a
complex interface.

217
TODO
View file

@ -1,217 +0,0 @@
High-priority:
Add a mechanism for forwarding callbacks between EventThread instances,
or even just between thread instances. Have a callback queue and an
fd to poll on, and when a remote thread adds something to an empty
callback queue, it writes something to the fd so as to wake up its
peer thread. NB: Even better if the poll-compatible wakeups (note,
that's the real point here: don't want to mix fd polling and cv waiting,
for instance) were dependent on the EventPoll implementation. That way
we could use kqueue's user types, etc., rather than needing to use a
pipe.
Alternately, move fd polling to its own thread and use condvars and
callback queues for everything else.
As part of testing/enabling/proofing that work, split EventThread into
two threads, and add one more. One thread which does FD polling. One
thread which does callback / timeout dispatch. And then move the I/O
system into its own thread. Does that sound about right? Easy then to
start to move other things to their own threads, because the threading
concerns of each of those subsystems would be covered. So we could
have some number of XCodec threads or whatever. A work submission
model that's fairly generic would be nice. So you could submit
callbacks or timeouts to the callback thread, fd registration and
deregestration to the poll thread, and I/O requests to the I/O thread.
Adds some latency, but probably worth it?
And for threads which need a specialized work submission process, we
could just override some method from Thread. Have a simple base class
for work items, let each type of thread do a dynamic_cast from there
up? Or provide more specific interfaces, do queueing in each thread's
class, and just provide a common alert() / biff() / ping() / wakeup()
method which alerts that an empty mailbox has become non-empty. So
the poll thread doesn't have to use condvars AND fds somehow. And let
each thread specify a timeout for sleeping, mostly useful for the
callback thread.
Make it so that Actions can be given a callback after they are created, and
then the source of the Action from that gets the hint to start and where to
continue after the asynchronous action. This has the nice benefit of getting
rid of the callback parameter to async methods, e.g.
handler_.wait(socket_->read(0, handler_.callback()));
Becomes:
handler_.wait(socket_->read(0));
Where handler_.wait is like:
void wait(Action *a)
{
a->callback(this, &EventHandler::handle_callback);
action_ = a;
}
Alternately, it would be nice to make asynchronous interfaces move to taking
CallbackHandler or EventHandler instances, e.g.
socket_->read(0, &handler_);
And then socket_->read is like:
void read(size_t amt, EventHandler *handler)
{
read_callback_ = handler->callback();
handler->wait(cancellation(this, Socket::read_cancel));
start_read();
}
I feel like continuing to live with these possibilities may make clear how the
various APIs may best change, e.g. perhaps read should be:
void read(size_t amt, EventHandler *handler)
{
read_handler_ = handler;
read_handler_->cancellation(this, Socket::read_cancel);
start_read();
}
...
void read_complete(Event e)
{
read_handler_->consume(e);
}
%%%
o) Do a pass with Log taking a LogHandle& not a const LogHandle& so I can
find places using hard-coded strings to move into LogHandles. Using a
LogHandle is vastly superior since it probably means doing the right
thing in virtual classes so that errors in the base class caused by the
subclass make it clear where the problem might be.
o) PipeProducer::input_do; PipeProducer::input_cork(), partial consume() ->
require input_cork().
Note:
o) Callbacks are going through a period of major change. For now, except for
the CallbackSchedulers, it is assumed that all callbacks will need to be
strongly typed, that is that most interfaces will take a SimpleCallback or
an EventCallback or similar, and use it directly. This will mostly be a
problem in the TimeoutQueue. If some class feels that it ought to be able
to schedule a timeout to handle a callback for, say, an EventCallback that
has already had its parameter set, we'll have problems.
NB: Once all extant code is converted in this manner, the CallbackBase
class could be named back to Callback, but since few things should be
using it directly, the more obtuse name may be desirable as an aid in
avoiding foot-shooting. Or just rename CallbackBase to Schedulable
and have done with it.
XXX Eventually the schedule function could be moved into SimpleCallback,
TypedCallback, etc., so that we can in the latter case blow up badly
if someone asks to schedule a function whose parameter has not been
set.
o) Make tack use the EventSystem and IOSystem now that the performance of
the latter is substantially better than the hand-rolled I/O of tack.
o) Replace singletons with thread-local storage.
o) Get simple packet capture/injection stuff working, enough to do some trivial
packet tunneling / deduplication stuff.
o) Packet framing, so we can divide incoming Buffers up into protocol control
and data fields, so that we deduplicate at useful boundaries and don't do
things like include ephemeral fields or sensitive information.
o) Begin introducing locking.
o) Split polling across multiple threads, or do callbacks in one thread and
run the IOSystem on another (IOThread?)
o) Find ways to reduce the cost of the EventThread abstraction, possibly by
decomposing it into several things and running timeouts and polling in
separate threads, so the inner loops are tighter. callback-speed1 has
gotten quite painful on Mac OS X (though little impact on FreeBSD, perhaps
thread-local storage is faster on FreeBSD?)
o) Add centralized implementations of Catenate and other patterns in lots of
the tests.
o) When splitting things into different threads, add a pipe-oriented condition
variable facility or something, at least for threads which need to poll,
rather than just use condition variabls. Or perhaps we should just use an
inter-thread messaging paradigm to handle different queues for each thread,
which is slow but we can probably batch updates through a scheduler.
For 0.7.1:
o) Add an ActionCache which classes can use to dole out Actions and which at
destruction time will assert that there are no outstanding actions. This
will make debugging code involving Actions easier and give better Action
allocation performance, done properly.
For 0.7.2:
o) Test error handling in epoll.
o) Test error handling with port(3C).
o) Make UnixClient and UnixServer take the SocketType as an argument to make
it possible to support both stream-orientation and datagram-orientation?
Would need to update UnixServer's API to support both models.
o) Something higher-level than Pipe which supports various disciplines and
makes it easier to write stream processing modules.
o) Make *Client::connect() work like TCPClient::connect().
o) Move to the getrusage-based Timer class and make it clear that that is what
it is for. Perhaps create a Timer base class and a UserTimer and WallTimer
for real use?
o) Add a flush token for Pipe::input() ?
o) Add a flush Event type for Pipe::output() ?
XXX It's unclear, but I think a flush method or similar may be needed to
implement buffer limiting? A flush method being like input(), except
that it only returns once the last Pipe in the Pipeline has returned
flush complete, or error if data cannot be flushed because there is not
enough to finish processing the data that is pending, at which point
you at least know that everything else has been flushed?
o) Make Pipe::input take a parameter to send EOS along with data.
o) How do we find out if remote has shut down read channel so we can stop
writing and shut down a Splice?
o) Make interface/listener objects automatically start listening.
o) Clean up address configuration objects to be more sensible, being either
specific components of addresses (e.g. to specify an IP address) or socket
addresses (e.g. AF_UNIX paths, IP+port pairs for TCP or UDP or SCTP.)
o) Connector abstraction: connection pooling / connecting via a SOCKS server.
For 0.8.0:
o) Cache hierarchy, including persistent storage.
o) TLS.
For 0.9.0:
o) Handle TCP OOB data.
After 1.0.0:
o) A CLI program for management. Remember to USE_POLL=select since Mac OS X can
only use select(2) for stdin and stdout.
o) Make it possible to detect when we are sending to a socket that is also
within WANProxy and avoid a system call -- just copy directly to the
appropriate buffer. It should be pretty easy to do this with the IO queueing
system if we getpeername/getsockname to identify this occurring.
o) Add an IO queueing system that will make it possible to use lio_listio on
systems that support it.
o) Make the XCodec encoder and decoder asynchronous.
o) Connection table. Database.
o) HTTP termination and reinitiation good enough to support an HTTP proxy mode.
Ongoing:
o) Add lots of comments.
o) Fix bugs.
o) Audit log levels.
o) Try to remove HALTs and NOTREACHEDs that aren't legitimate.
o) Give better debug information from configuration system.
Maybe:
o) A resolver that's less fragile than the OS-supplied ones. Mac OS X, at
minimum, neither keeps a pool of file descriptors nor errors out gracefully
when the OS is out of them, leading to hangs.
o) Send definitions out-of-band, too, so that QoS and backpressure on one
connection can't delay other connections.
o) In-path forwarding using BPF and a tiny network stack.
o) Run-length-encoding.
o) Many compression algorithms.
o) Allow chaining codecs.
o) SOCKS IPv6 support.
o) Some decent way to configure Pipelines.
o) Figure out a good name for a Pipeline, since Pipeline seems rubbish.
o) Merge two Pipelines.
o) Convert the SOCKS proxy server to a PipeEndpoint that merges the Pipeline
that it is connected to with a newly-created one.
o) Be less protocol-ignorant; add protocol-aware framing to WANProxy. For
example, HTTP response headers are likely to include some amount of changing
data (timestamps, etc.) so perhaps it's better to take a clean shot at the
start of the content. And perhaps it's better to convert to large chunks in
chunked encoding mode to get bigger windows of data to encode. No need to
remember the last 3 bytes of a chunk, the chunk header, and the next N bytes,
if the chunks won't be laid out the same every time. Right?

57
sample.conf Normal file
View file

@ -0,0 +1,57 @@
###############################################################################
#
# Sample configuration file for WANProxy XTech v3.0
#
# Codec definition must include following cache directives:
# - cache: Memory (default) or COSS (use persistent cache in disk)
# - cache_path: location for the cache files (if using COSS)
# - local_size: size in MB for the local cache of the encoder. The decoder
# will receive this value on the other side and use it for
# its own cache, so the old parameter remote_size is no
# longer needed and should not be used any more.
#
# Proxy definition can include an additional informative parameter:
# - role: Client (originates requests) or Server. When not specified,
# a proxy taking unencoded input and writing encoded output
# is considered to be a client.
#
# Only one proxy may be defined per instance. To host several proxies
# on the same node different processes must be launched each one with
# its own configuration.
#
# Upon reception of SIGHUP the daemon will reread this file and apply the
# new values to any subsequent connections.
#
###############################################################################
create codec codec0
set codec0.codec XCodec
set codec0.cache COSS
set codec0.cache_path "/var/lib/wanproxy"
set codec0.local_size 2048
set codec0.compressor zlib
set codec0.compressor_level 6
activate codec0
create interface if0
set if0.family IPv4
set if0.host "192.168.1.23"
set if0.port "3301"
set if0.proto TCP
activate if0
create peer peer0
set peer0.family IPv4
set peer0.host "localhost"
set peer0.port "3128"
set peer0.proto TCP
activate peer0
create proxy proxy0
set proxy0.interface if0
set proxy0.interface_codec codec0
set proxy0.peer peer0
set proxy0.peer_codec None
set proxy0.role Server
activate proxy0