Commit 9f1bd073 authored by Andy Goldstein's avatar Andy Goldstein Committed by Steve Milner

Add goproxy test image

parent e7365241
...@@ -214,6 +214,11 @@ ...@@ -214,6 +214,11 @@
"Rev": "3dcc96556217539f50599357fb481ac0dc7439b9" "Rev": "3dcc96556217539f50599357fb481ac0dc7439b9"
}, },
{ {
"ImportPath": "github.com/elazarl/goproxy",
"Comment": "v1.0-66-g07b16b6",
"Rev": "07b16b6e30fcac0ad8c0435548e743bcf2ca7e92"
},
{
"ImportPath": "github.com/emicklei/go-restful", "ImportPath": "github.com/emicklei/go-restful",
"Comment": "v1.1.3-98-g1f9a0ee", "Comment": "v1.1.3-98-g1f9a0ee",
"Rev": "1f9a0ee00ff93717a275e15b30cf7df356255877" "Rev": "1f9a0ee00ff93717a275e15b30cf7df356255877"
......
Copyright (c) 2012 Elazar Leibovich. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.
* Neither the name of Elazar Leibovich. nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
# Introduction
[![Join the chat at https://gitter.im/elazarl/goproxy](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/elazarl/goproxy?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
Package goproxy provides a customizable HTTP proxy library for Go (golang),
It supports regular HTTP proxy, HTTPS through CONNECT, and "hijacking" HTTPS
connection using "Man in the Middle" style attack.
The intent of the proxy, is to be usable with reasonable amount of traffic
yet, customizable and programable.
The proxy itself is simply a `net/http` handler.
In order to use goproxy, one should set their browser to use goproxy as an HTTP
proxy. Here is how you do that [in Chrome](https://support.google.com/chrome/answer/96815?hl=en)
and [in Firefox](http://www.wikihow.com/Enter-Proxy-Settings-in-Firefox).
For example, the URL you should use as proxy when running `./bin/basic` is
`localhost:8080`, as this is the default binding for the basic proxy.
## Mailing List
New features would be discussed on the [mailing list](https://groups.google.com/forum/#!forum/goproxy-dev)
before their development.
## Latest Stable Release
Get the latest goproxy from `gopkg.in/elazarl/goproxy.v1`.
# Why not Fiddler2?
Fiddler is an excellent software with similar intent. However, Fiddler is not
as customable as goproxy intend to be. The main difference is, Fiddler is not
intended to be used as a real proxy.
A possible use case that suits goproxy but
not Fiddler, is, gathering statisitics on page load times for a certain website over a week.
With goproxy you could ask all your users to set their proxy to a dedicated machine running a
goproxy server. Fiddler is a GUI app not designed to be ran like a server for multiple users.
# A taste of goproxy
To get a taste of `goproxy`, a basic HTTP/HTTPS transparent proxy
import (
"github.com/elazarl/goproxy"
"log"
"net/http"
)
func main() {
proxy := goproxy.NewProxyHttpServer()
proxy.Verbose = true
log.Fatal(http.ListenAndServe(":8080", proxy))
}
This line will add `X-GoProxy: yxorPoG-X` header to all requests sent through the proxy
proxy.OnRequest().DoFunc(
func(r *http.Request,ctx *goproxy.ProxyCtx)(*http.Request,*http.Response) {
r.Header.Set("X-GoProxy","yxorPoG-X")
return r,nil
})
`DoFunc` will process all incoming requests to the proxy. It will add a header to the request
and return it. The proxy will send the modified request.
Note that we returned nil value as the response. Have we returned a response, goproxy would
have discarded the request and sent the new response to the client.
In order to refuse connections to reddit at work time
proxy.OnRequest(goproxy.DstHostIs("www.reddit.com")).DoFunc(
func(r *http.Request,ctx *goproxy.ProxyCtx)(*http.Request,*http.Response) {
if h,_,_ := time.Now().Clock(); h >= 8 && h <= 17 {
return r,goproxy.NewResponse(r,
goproxy.ContentTypeText,http.StatusForbidden,
"Don't waste your time!")
}
return r,nil
})
`DstHostIs` returns a `ReqCondition`, that is a function receiving a `Request` and returning a boolean
we will only process requests that matches the condition. `DstHostIs("www.reddit.com")` will return
a `ReqCondition` accepting only requests directed to "www.reddit.com".
`DoFunc` will recieve a function that will preprocess the request. We can change the request, or
return a response. If the time is between 8:00am and 17:00pm, we will neglect the request, and
return a precanned text response saying "do not waste your time".
See additional examples in the examples directory.
# What's New
1. Ability to `Hijack` CONNECT requests. See
[the eavesdropper example](https://github.com/elazarl/goproxy/blob/master/examples/goproxy-eavesdropper/main.go#L27)
2. Transparent proxy support for http/https including MITM certificate generation for TLS. See the [transparent example.](https://github.com/elazarl/goproxy/tree/master/examples/goproxy-transparent)
# License
I put the software temporarily under the Go-compatible BSD license,
if this prevents someone from using the software, do let mee know and I'll consider changing it.
At any rate, user feedback is very important for me, so I'll be delighted to know if you're using this package.
# Beta Software
I've received a positive feedback from a few people who use goproxy in production settings.
I believe it is good enough for usage.
I'll try to keep reasonable backwards compatability. In case of a major API change,
I'll change the import path.
package goproxy
import "net/http"
// ReqHandler will "tamper" with the request coming to the proxy server
// If Handle returns req,nil the proxy will send the returned request
// to the destination server. If it returns nil,resp the proxy will
// skip sending any requests, and will simply return the response `resp`
// to the client.
type ReqHandler interface {
Handle(req *http.Request, ctx *ProxyCtx) (*http.Request, *http.Response)
}
// A wrapper that would convert a function to a ReqHandler interface type
type FuncReqHandler func(req *http.Request, ctx *ProxyCtx) (*http.Request, *http.Response)
// FuncReqHandler.Handle(req,ctx) <=> FuncReqHandler(req,ctx)
func (f FuncReqHandler) Handle(req *http.Request, ctx *ProxyCtx) (*http.Request, *http.Response) {
return f(req, ctx)
}
// after the proxy have sent the request to the destination server, it will
// "filter" the response through the RespHandlers it has.
// The proxy server will send to the client the response returned by the RespHandler.
// In case of error, resp will be nil, and ctx.RoundTrip.Error will contain the error
type RespHandler interface {
Handle(resp *http.Response, ctx *ProxyCtx) *http.Response
}
// A wrapper that would convert a function to a RespHandler interface type
type FuncRespHandler func(resp *http.Response, ctx *ProxyCtx) *http.Response
// FuncRespHandler.Handle(req,ctx) <=> FuncRespHandler(req,ctx)
func (f FuncRespHandler) Handle(resp *http.Response, ctx *ProxyCtx) *http.Response {
return f(resp, ctx)
}
// When a client send a CONNECT request to a host, the request is filtered through
// all the HttpsHandlers the proxy has, and if one returns true, the connection is
// sniffed using Man in the Middle attack.
// That is, the proxy will create a TLS connection with the client, another TLS
// connection with the destination the client wished to connect to, and would
// send back and forth all messages from the server to the client and vice versa.
// The request and responses sent in this Man In the Middle channel are filtered
// through the usual flow (request and response filtered through the ReqHandlers
// and RespHandlers)
type HttpsHandler interface {
HandleConnect(req string, ctx *ProxyCtx) (*ConnectAction, string)
}
// A wrapper that would convert a function to a HttpsHandler interface type
type FuncHttpsHandler func(host string, ctx *ProxyCtx) (*ConnectAction, string)
// FuncHttpsHandler should implement the RespHandler interface
func (f FuncHttpsHandler) HandleConnect(host string, ctx *ProxyCtx) (*ConnectAction, string) {
return f(host, ctx)
}
#!/bin/bash
go test || exit
for action in $@; do go $action; done
mkdir -p bin
find regretable examples/* ext/* -maxdepth 0 -type d | while read d; do
(cd $d
go build -o ../../bin/$(basename $d)
find *_test.go -maxdepth 0 2>/dev/null|while read f;do
for action in $@; do go $action; done
go test
break
done)
done
-----BEGIN CERTIFICATE-----
MIICSjCCAbWgAwIBAgIBADALBgkqhkiG9w0BAQUwSjEjMCEGA1UEChMaZ2l0aHVi
LmNvbS9lbGF6YXJsL2dvcHJveHkxIzAhBgNVBAMTGmdpdGh1Yi5jb20vZWxhemFy
bC9nb3Byb3h5MB4XDTAwMDEwMTAwMDAwMFoXDTQ5MTIzMTIzNTk1OVowSjEjMCEG
A1UEChMaZ2l0aHViLmNvbS9lbGF6YXJsL2dvcHJveHkxIzAhBgNVBAMTGmdpdGh1
Yi5jb20vZWxhemFybC9nb3Byb3h5MIGdMAsGCSqGSIb3DQEBAQOBjQAwgYkCgYEA
vz9BbCaJjxs73Tvcq3leP32hAGerQ1RgvlZ68Z4nZmoVHfl+2Nr/m0dmW+GdOfpT
cs/KzfJjYGr/84x524fiuR8GdZ0HOtXJzyF5seoWnbBIuyr1PbEpgRhGQMqqOUuj
YExeLbfNHPIoJ8XZ1Vzyv3YxjbmjWA+S/uOe9HWtDbMCAwEAAaNGMEQwDgYDVR0P
AQH/BAQDAgCkMBMGA1UdJQQMMAoGCCsGAQUFBwMBMA8GA1UdEwEB/wQFMAMBAf8w
DAYDVR0RBAUwA4IBKjALBgkqhkiG9w0BAQUDgYEAIcL8huSmGMompNujsvePTUnM
oEUKtX4Eh/+s+DSfV/TyI0I+3GiPpLplEgFWuoBIJGios0r1dKh5N0TGjxX/RmGm
qo7E4jjJuo8Gs5U8/fgThZmshax2lwLtbRNwhvUVr65GdahLsZz8I+hySLuatVvR
qHHq/FQORIiNyNpq/Hg=
-----END CERTIFICATE-----
package goproxy
import (
"crypto/tls"
"crypto/x509"
)
func init() {
if goproxyCaErr != nil {
panic("Error parsing builtin CA " + goproxyCaErr.Error())
}
var err error
if GoproxyCa.Leaf, err = x509.ParseCertificate(GoproxyCa.Certificate[0]); err != nil {
panic("Error parsing builtin CA " + err.Error())
}
}
var tlsClientSkipVerify = &tls.Config{InsecureSkipVerify: true}
var defaultTLSConfig = &tls.Config{
InsecureSkipVerify: true,
}
var CA_CERT = []byte(`-----BEGIN CERTIFICATE-----
MIICSjCCAbWgAwIBAgIBADALBgkqhkiG9w0BAQUwSjEjMCEGA1UEChMaZ2l0aHVi
LmNvbS9lbGF6YXJsL2dvcHJveHkxIzAhBgNVBAMTGmdpdGh1Yi5jb20vZWxhemFy
bC9nb3Byb3h5MB4XDTAwMDEwMTAwMDAwMFoXDTQ5MTIzMTIzNTk1OVowSjEjMCEG
A1UEChMaZ2l0aHViLmNvbS9lbGF6YXJsL2dvcHJveHkxIzAhBgNVBAMTGmdpdGh1
Yi5jb20vZWxhemFybC9nb3Byb3h5MIGdMAsGCSqGSIb3DQEBAQOBjQAwgYkCgYEA
vz9BbCaJjxs73Tvcq3leP32hAGerQ1RgvlZ68Z4nZmoVHfl+2Nr/m0dmW+GdOfpT
cs/KzfJjYGr/84x524fiuR8GdZ0HOtXJzyF5seoWnbBIuyr1PbEpgRhGQMqqOUuj
YExeLbfNHPIoJ8XZ1Vzyv3YxjbmjWA+S/uOe9HWtDbMCAwEAAaNGMEQwDgYDVR0P
AQH/BAQDAgCkMBMGA1UdJQQMMAoGCCsGAQUFBwMBMA8GA1UdEwEB/wQFMAMBAf8w
DAYDVR0RBAUwA4IBKjALBgkqhkiG9w0BAQUDgYEAIcL8huSmGMompNujsvePTUnM
oEUKtX4Eh/+s+DSfV/TyI0I+3GiPpLplEgFWuoBIJGios0r1dKh5N0TGjxX/RmGm
qo7E4jjJuo8Gs5U8/fgThZmshax2lwLtbRNwhvUVr65GdahLsZz8I+hySLuatVvR
qHHq/FQORIiNyNpq/Hg=
-----END CERTIFICATE-----`)
var CA_KEY = []byte(`-----BEGIN RSA PRIVATE KEY-----
MIICXQIBAAKBgQC/P0FsJomPGzvdO9yreV4/faEAZ6tDVGC+VnrxnidmahUd+X7Y
2v+bR2Zb4Z05+lNyz8rN8mNgav/zjHnbh+K5HwZ1nQc61cnPIXmx6hadsEi7KvU9
sSmBGEZAyqo5S6NgTF4tt80c8ignxdnVXPK/djGNuaNYD5L+4570da0NswIDAQAB
AoGBALzIv1b4D7ARTR3NOr6V9wArjiOtMjUrdLhO+9vIp9IEA8ZsA9gjDlCEwbkP
VDnoLjnWfraff5Os6+3JjHy1fYpUiCdnk2XA6iJSL1XWKQZPt3wOunxP4lalDgED
QTRReFbA/y/Z4kSfTXpVj68ytcvSRW/N7q5/qRtbN9804jpBAkEA0s6lvH2btSLA
mcEdwhs7zAslLbdld7rvfUeP82gPPk0S6yUqTNyikqshM9AwAktHY7WvYdKl+ghZ
HTxKVC4DoQJBAOg/IAW5RbXknP+Lf7AVtBgw3E+Yfa3mcdLySe8hjxxyZq825Zmu
Rt5Qj4Lw6ifSFNy4kiiSpE/ZCukYvUXGENMCQFkPxSWlS6tzSzuqQxBGwTSrYMG3
wb6b06JyIXcMd6Qym9OMmBpw/J5KfnSNeDr/4uFVWQtTG5xO+pdHaX+3EQECQQDl
qcbY4iX1gWVfr2tNjajSYz751yoxVbkpiT9joiQLVXYFvpu+JYEfRzsjmWl0h2Lq
AftG8/xYmaEYcMZ6wSrRAkBUwiom98/8wZVlB6qbwhU1EKDFANvICGSWMIhPx3v7
MJqTIj4uJhte2/uyVvZ6DC6noWYgy+kLgqG0S97tUEG8
-----END RSA PRIVATE KEY-----`)
var GoproxyCa, goproxyCaErr = tls.X509KeyPair(CA_CERT, CA_KEY)
// Taken from $GOROOT/src/pkg/net/http/chunked
// needed to write https responses to client.
package goproxy
import (
"io"
"strconv"
)
// newChunkedWriter returns a new chunkedWriter that translates writes into HTTP
// "chunked" format before writing them to w. Closing the returned chunkedWriter
// sends the final 0-length chunk that marks the end of the stream.
//
// newChunkedWriter is not needed by normal applications. The http
// package adds chunking automatically if handlers don't set a
// Content-Length header. Using newChunkedWriter inside a handler
// would result in double chunking or chunking with a Content-Length
// length, both of which are wrong.
func newChunkedWriter(w io.Writer) io.WriteCloser {
return &chunkedWriter{w}
}
// Writing to chunkedWriter translates to writing in HTTP chunked Transfer
// Encoding wire format to the underlying Wire chunkedWriter.
type chunkedWriter struct {
Wire io.Writer
}
// Write the contents of data as one chunk to Wire.
// NOTE: Note that the corresponding chunk-writing procedure in Conn.Write has
// a bug since it does not check for success of io.WriteString
func (cw *chunkedWriter) Write(data []byte) (n int, err error) {
// Don't send 0-length data. It looks like EOF for chunked encoding.
if len(data) == 0 {
return 0, nil
}
head := strconv.FormatInt(int64(len(data)), 16) + "\r\n"
if _, err = io.WriteString(cw.Wire, head); err != nil {
return 0, err
}
if n, err = cw.Wire.Write(data); err != nil {
return
}
if n != len(data) {
err = io.ErrShortWrite
return
}
_, err = io.WriteString(cw.Wire, "\r\n")
return
}
func (cw *chunkedWriter) Close() error {
_, err := io.WriteString(cw.Wire, "0\r\n")
return err
}
package goproxy
import (
"crypto/aes"
"crypto/cipher"
"crypto/rsa"
"crypto/sha256"
"crypto/x509"
"errors"
)
type CounterEncryptorRand struct {
cipher cipher.Block
counter []byte
rand []byte
ix int
}
func NewCounterEncryptorRandFromKey(key interface{}, seed []byte) (r CounterEncryptorRand, err error) {
var keyBytes []byte
switch key := key.(type) {
case *rsa.PrivateKey:
keyBytes = x509.MarshalPKCS1PrivateKey(key)
default:
err = errors.New("only RSA keys supported")
return
}
h := sha256.New()
if r.cipher, err = aes.NewCipher(h.Sum(keyBytes)[:aes.BlockSize]); err != nil {
return
}
r.counter = make([]byte, r.cipher.BlockSize())
if seed != nil {
copy(r.counter, h.Sum(seed)[:r.cipher.BlockSize()])
}
r.rand = make([]byte, r.cipher.BlockSize())
r.ix = len(r.rand)
return
}
func (c *CounterEncryptorRand) Seed(b []byte) {
if len(b) != len(c.counter) {
panic("SetCounter: wrong counter size")
}
copy(c.counter, b)
}
func (c *CounterEncryptorRand) refill() {
c.cipher.Encrypt(c.rand, c.counter)
for i := 0; i < len(c.counter); i++ {
if c.counter[i]++; c.counter[i] != 0 {
break
}
}
c.ix = 0
}
func (c *CounterEncryptorRand) Read(b []byte) (n int, err error) {
if c.ix == len(c.rand) {
c.refill()
}
if n = len(c.rand) - c.ix; n > len(b) {
n = len(b)
}
copy(b, c.rand[c.ix:c.ix+n])
c.ix += n
return
}
package goproxy_test
import (
"bytes"
"crypto/rsa"
"encoding/binary"
"github.com/elazarl/goproxy"
"io"
"math"
"math/rand"
"testing"
)
type RandSeedReader struct {
r rand.Rand
}
func (r *RandSeedReader) Read(b []byte) (n int, err error) {
for i := range b {
b[i] = byte(r.r.Int() & 0xFF)
}
return len(b), nil
}
func TestCounterEncDifferentConsecutive(t *testing.T) {
k, err := rsa.GenerateKey(&RandSeedReader{*rand.New(rand.NewSource(0xFF43109))}, 128)
fatalOnErr(err, "rsa.GenerateKey", t)
c, err := goproxy.NewCounterEncryptorRandFromKey(k, []byte("the quick brown fox run over the lazy dog"))
fatalOnErr(err, "NewCounterEncryptorRandFromKey", t)
for i := 0; i < 100*1000; i++ {
var a, b int64
binary.Read(&c, binary.BigEndian, &a)
binary.Read(&c, binary.BigEndian, &b)
if a == b {
t.Fatal("two consecutive equal int64", a, b)
}
}
}
func TestCounterEncIdenticalStreams(t *testing.T) {
k, err := rsa.GenerateKey(&RandSeedReader{*rand.New(rand.NewSource(0xFF43109))}, 128)
fatalOnErr(err, "rsa.GenerateKey", t)
c1, err := goproxy.NewCounterEncryptorRandFromKey(k, []byte("the quick brown fox run over the lazy dog"))
fatalOnErr(err, "NewCounterEncryptorRandFromKey", t)
c2, err := goproxy.NewCounterEncryptorRandFromKey(k, []byte("the quick brown fox run over the lazy dog"))
fatalOnErr(err, "NewCounterEncryptorRandFromKey", t)
nout := 1000
out1, out2 := make([]byte, nout), make([]byte, nout)
io.ReadFull(&c1, out1)
tmp := out2[:]
rand.Seed(0xFF43109)
for len(tmp) > 0 {
n := 1 + rand.Intn(256)
if n > len(tmp) {
n = len(tmp)
}
n, err := c2.Read(tmp[:n])
fatalOnErr(err, "CounterEncryptorRand.Read", t)
tmp = tmp[n:]
}
if !bytes.Equal(out1, out2) {
t.Error("identical CSPRNG does not produce the same output")
}
}
func stddev(data []int) float64 {
var sum, sum_sqr float64 = 0, 0
for _, h := range data {
sum += float64(h)
sum_sqr += float64(h) * float64(h)
}
n := float64(len(data))
variance := (sum_sqr - ((sum * sum) / n)) / (n - 1)
return math.Sqrt(variance)
}
func TestCounterEncStreamHistogram(t *testing.T) {
k, err := rsa.GenerateKey(&RandSeedReader{*rand.New(rand.NewSource(0xFF43109))}, 128)
fatalOnErr(err, "rsa.GenerateKey", t)
c, err := goproxy.NewCounterEncryptorRandFromKey(k, []byte("the quick brown fox run over the lazy dog"))
fatalOnErr(err, "NewCounterEncryptorRandFromKey", t)
nout := 100 * 1000
out := make([]byte, nout)
io.ReadFull(&c, out)
refhist := make([]int, 256)
for i := 0; i < nout; i++ {
refhist[rand.Intn(256)]++
}
hist := make([]int, 256)
for _, b := range out {
hist[int(b)]++
}
refstddev, stddev := stddev(refhist), stddev(hist)
// due to lack of time, I guestimate
t.Logf("ref:%v - act:%v = %v", refstddev, stddev, math.Abs(refstddev-stddev))
if math.Abs(refstddev-stddev) >= 1 {
t.Errorf("stddev of ref histogram different than regular PRNG: %v %v", refstddev, stddev)
}
}
package goproxy
import (
"net/http"
"regexp"
)
// ProxyCtx is the Proxy context, contains useful information about every request. It is passed to
// every user function. Also used as a logger.
type ProxyCtx struct {
// Will contain the client request from the proxy
Req *http.Request
// Will contain the remote server's response (if available. nil if the request wasn't send yet)
Resp *http.Response
RoundTripper RoundTripper
// will contain the recent error that occured while trying to send receive or parse traffic
Error error
// A handle for the user to keep data in the context, from the call of ReqHandler to the
// call of RespHandler
UserData interface{}
// Will connect a request to a response
Session int64
proxy *ProxyHttpServer
}
type RoundTripper interface {
RoundTrip(req *http.Request, ctx *ProxyCtx) (*http.Response, error)
}
type RoundTripperFunc func(req *http.Request, ctx *ProxyCtx) (*http.Response, error)
func (f RoundTripperFunc) RoundTrip(req *http.Request, ctx *ProxyCtx) (*http.Response, error) {
return f(req, ctx)
}
func (ctx *ProxyCtx) RoundTrip(req *http.Request) (*http.Response, error) {
if ctx.RoundTripper != nil {
return ctx.RoundTripper.RoundTrip(req, ctx)
}
return ctx.proxy.Tr.RoundTrip(req)
}
func (ctx *ProxyCtx) printf(msg string, argv ...interface{}) {
ctx.proxy.Logger.Printf("[%03d] "+msg+"\n", append([]interface{}{ctx.Session & 0xFF}, argv...)...)
}
// Logf prints a message to the proxy's log. Should be used in a ProxyHttpServer's filter
// This message will be printed only if the Verbose field of the ProxyHttpServer is set to true
//
// proxy.OnRequest().DoFunc(func(r *http.Request,ctx *goproxy.ProxyCtx) (*http.Request, *http.Response){
// nr := atomic.AddInt32(&counter,1)
// ctx.Printf("So far %d requests",nr)
// return r, nil
// })
func (ctx *ProxyCtx) Logf(msg string, argv ...interface{}) {
if ctx.proxy.Verbose {
ctx.printf("INFO: "+msg, argv...)
}
}
// Warnf prints a message to the proxy's log. Should be used in a ProxyHttpServer's filter
// This message will always be printed.
//
// proxy.OnRequest().DoFunc(func(r *http.Request,ctx *goproxy.ProxyCtx) (*http.Request, *http.Response){
// f,err := os.OpenFile(cachedContent)
// if err != nil {
// ctx.Warnf("error open file %v: %v",cachedContent,err)
// return r, nil
// }
// return r, nil
// })
func (ctx *ProxyCtx) Warnf(msg string, argv ...interface{}) {
ctx.printf("WARN: "+msg, argv...)
}
var charsetFinder = regexp.MustCompile("charset=([^ ;]*)")
// Will try to infer the character set of the request from the headers.
// Returns the empty string if we don't know which character set it used.
// Currently it will look for charset=<charset> in the Content-Type header of the request.
func (ctx *ProxyCtx) Charset() string {
charsets := charsetFinder.FindStringSubmatch(ctx.Resp.Header.Get("Content-Type"))
if charsets == nil {
return ""
}
return charsets[1]
}
/*
Package goproxy provides a customizable HTTP proxy,
supporting hijacking HTTPS connection.
The intent of the proxy, is to be usable with reasonable amount of traffic
yet, customizable and programable.
The proxy itself is simply an `net/http` handler.
Typical usage is
proxy := goproxy.NewProxyHttpServer()
proxy.OnRequest(..conditions..).Do(..requesthandler..)
proxy.OnRequest(..conditions..).DoFunc(..requesthandlerFunction..)
proxy.OnResponse(..conditions..).Do(..responesHandler..)
proxy.OnResponse(..conditions..).DoFunc(..responesHandlerFunction..)
http.ListenAndServe(":8080", proxy)
Adding a header to each request
proxy.OnRequest().DoFunc(func(r *http.Request,ctx *goproxy.ProxyCtx) (*http.Request, *http.Response){
r.Header.Set("X-GoProxy","1")
return r, nil
})
Note that the function is called before the proxy sends the request to the server
For printing the content type of all incoming responses
proxy.OnResponse().DoFunc(func(r *http.Response, ctx *goproxy.ProxyCtx)*http.Response{
println(ctx.Req.Host,"->",r.Header.Get("Content-Type"))
return r
})
note that we used the ProxyCtx context variable here. It contains the request
and the response (Req and Resp, Resp is nil if unavailable) of this specific client
interaction with the proxy.
To print the content type of all responses from a certain url, we'll add a
ReqCondition to the OnResponse function:
proxy.OnResponse(goproxy.UrlIs("golang.org/pkg")).DoFunc(func(r *http.Response, ctx *goproxy.ProxyCtx)*http.Response{
println(ctx.Req.Host,"->",r.Header.Get("Content-Type"))
return r
})
We can write the condition ourselves, conditions can be set on request and on response
var random = ReqConditionFunc(func(r *http.Request) bool {
return rand.Intn(1) == 0
})
var hasGoProxyHeader = RespConditionFunc(func(resp *http.Response,req *http.Request)bool {
return resp.Header.Get("X-GoProxy") != ""
})
Caution! If you give a RespCondition to the OnRequest function, you'll get a run time panic! It doesn't
make sense to read the response, if you still haven't got it!
Finally, we have convenience function to throw a quick response
proxy.OnResponse(hasGoProxyHeader).DoFunc(func(r*http.Response,ctx *goproxy.ProxyCtx)*http.Response {
r.Body.Close()
return goproxy.ForbiddenTextResponse(ctx.Req,"Can't see response with X-GoProxy header!")
})
we close the body of the original repsonse, and return a new 403 response with a short message.
Example use cases:
1. https://github.com/elazarl/goproxy/tree/master/examples/goproxy-avgsize
To measure the average size of an Html served in your site. One can ask
all the QA team to access the website by a proxy, and the proxy will
measure the average size of all text/html responses from your host.
2. [not yet implemented]
All requests to your web servers should be directed through the proxy,
when the proxy will detect html pieces sent as a response to AJAX
request, it'll send a warning email.
3. https://github.com/elazarl/goproxy/blob/master/examples/goproxy-httpdump/
Generate a real traffic to your website by real users using through
proxy. Record the traffic, and try it again for more real load testing.
4. https://github.com/elazarl/goproxy/tree/master/examples/goproxy-no-reddit-at-worktime
Will allow browsing to reddit.com between 8:00am and 17:00pm
5. https://github.com/elazarl/goproxy/tree/master/examples/goproxy-jquery-version
Will warn if multiple versions of jquery are used in the same domain.
6. https://github.com/elazarl/goproxy/blob/master/examples/goproxy-upside-down-ternet/
Modifies image files in an HTTP response via goproxy's image extension found in ext/.
*/
package goproxy
# Simple HTTP Proxy
`goproxy-basic` starts an HTTP proxy on :8080. It only handles explicit CONNECT
requests.
Start it in one shell:
```sh
goproxy-basic -v
```
Fetch goproxy homepage in another:
```sh
http_proxy=http://127.0.0.1:8080 wget -O - \
http://ripper234.com/p/introducing-goproxy-light-http-proxy/
```
The homepage HTML content should be displayed in the console. The proxy should
have logged the request being processed:
```sh
2015/04/09 18:19:17 [001] INFO: Got request /p/introducing-goproxy-light-http-proxy/ ripper234.com GET http://ripper234.com/p/introducing-goproxy-light-http-proxy/
2015/04/09 18:19:17 [001] INFO: Sending request GET http://ripper234.com/p/introducing-goproxy-light-http-proxy/
2015/04/09 18:19:18 [001] INFO: Received response 200 OK
2015/04/09 18:19:18 [001] INFO: Copying response to client 200 OK [200]
2015/04/09 18:19:18 [001] INFO: Copied 44333 bytes to client error=<nil>
```
package main
import (
"github.com/elazarl/goproxy"
"log"
"flag"
"net/http"
)
func main() {
verbose := flag.Bool("v", false, "should every proxy request be logged to stdout")
addr := flag.String("addr", ":8080", "proxy listen address")
flag.Parse()
proxy := goproxy.NewProxyHttpServer()
proxy.Verbose = *verbose
log.Fatal(http.ListenAndServe(*addr, proxy))
}
package main
import (
"bufio"
"flag"
"log"
"net"
"net/http"
"regexp"
"github.com/elazarl/goproxy"
)
func orPanic(err error) {
if err != nil {
panic(err)
}
}
func main() {
proxy := goproxy.NewProxyHttpServer()
proxy.OnRequest(goproxy.ReqHostMatches(regexp.MustCompile("^.*baidu.com$"))).
HandleConnect(goproxy.AlwaysReject)
proxy.OnRequest(goproxy.ReqHostMatches(regexp.MustCompile("^.*$"))).
HandleConnect(goproxy.AlwaysMitm)
// enable curl -p for all hosts on port 80
proxy.OnRequest(goproxy.ReqHostMatches(regexp.MustCompile("^.*:80$"))).
HijackConnect(func(req *http.Request, client net.Conn, ctx *goproxy.ProxyCtx) {
defer func() {
if e := recover(); e != nil {
ctx.Logf("error connecting to remote: %v", e)
client.Write([]byte("HTTP/1.1 500 Cannot reach destination\r\n\r\n"))
}
client.Close()
}()
clientBuf := bufio.NewReadWriter(bufio.NewReader(client), bufio.NewWriter(client))
remote, err := net.Dial("tcp", req.URL.Host)
orPanic(err)
remoteBuf := bufio.NewReadWriter(bufio.NewReader(remote), bufio.NewWriter(remote))
for {
req, err := http.ReadRequest(clientBuf.Reader)
orPanic(err)
orPanic(req.Write(remoteBuf))
orPanic(remoteBuf.Flush())
resp, err := http.ReadResponse(remoteBuf.Reader, req)
orPanic(err)
orPanic(resp.Write(clientBuf.Writer))
orPanic(clientBuf.Flush())
}
})
verbose := flag.Bool("v", false, "should every proxy request be logged to stdout")
addr := flag.String("addr", ":8080", "proxy listen address")
flag.Parse()
proxy.Verbose = *verbose
log.Fatal(http.ListenAndServe(*addr, proxy))
}
# Trace HTTP Requests and Responses
`goproxy-httpdump` starts an HTTP proxy on :8080. It handles explicit CONNECT
requests and traces them in a "db" directory created in the proxy working
directory. Each request type and headers are logged in a "log" file, while
their bodies are dumped in files prefixed with the request session identifier.
Additionally, the example demonstrates how to:
- Log information asynchronously (see HttpLogger)
- Allow the proxy to be stopped manually while ensuring all pending requests
have been processed (in this case, logged).
Start it in one shell:
```sh
goproxy-httpdump
```
Fetch goproxy homepage in another:
```sh
http_proxy=http://127.0.0.1:8080 wget -O - \
http://ripper234.com/p/introducing-goproxy-light-http-proxy/
```
A "db" directory should have appeared where you started the proxy, containing
two files:
- log: the request/response traces
- 1\_resp: the first response body
package main
import (
"errors"
"flag"
"fmt"
"io"
"log"
"net"
"net/http"
"net/http/httputil"
"os"
"os/signal"
"path"
"sync"
"time"
"github.com/elazarl/goproxy"
"github.com/elazarl/goproxy/transport"
)
type FileStream struct {
path string
f *os.File
}
func NewFileStream(path string) *FileStream {
return &FileStream{path, nil}
}
func (fs *FileStream) Write(b []byte) (nr int, err error) {
if fs.f == nil {
fs.f, err = os.Create(fs.path)
if err != nil {
return 0, err
}
}
return fs.f.Write(b)
}
func (fs *FileStream) Close() error {
fmt.Println("Close", fs.path)
if fs.f == nil {
return errors.New("FileStream was never written into")
}
return fs.f.Close()
}
type Meta struct {
req *http.Request
resp *http.Response
err error
t time.Time
sess int64
bodyPath string
from string
}
func fprintf(nr *int64, err *error, w io.Writer, pat string, a ...interface{}) {
if *err != nil {
return
}
var n int
n, *err = fmt.Fprintf(w, pat, a...)
*nr += int64(n)
}
func write(nr *int64, err *error, w io.Writer, b []byte) {
if *err != nil {
return
}
var n int
n, *err = w.Write(b)
*nr += int64(n)
}
func (m *Meta) WriteTo(w io.Writer) (nr int64, err error) {
if m.req != nil {
fprintf(&nr, &err, w, "Type: request\r\n")
} else if m.resp != nil {
fprintf(&nr, &err, w, "Type: response\r\n")
}
fprintf(&nr, &err, w, "ReceivedAt: %v\r\n", m.t)
fprintf(&nr, &err, w, "Session: %d\r\n", m.sess)
fprintf(&nr, &err, w, "From: %v\r\n", m.from)
if m.err != nil {
// note the empty response
fprintf(&nr, &err, w, "Error: %v\r\n\r\n\r\n\r\n", m.err)
} else if m.req != nil {
fprintf(&nr, &err, w, "\r\n")
buf, err2 := httputil.DumpRequest(m.req, false)
if err2 != nil {
return nr, err2
}
write(&nr, &err, w, buf)
} else if m.resp != nil {
fprintf(&nr, &err, w, "\r\n")
buf, err2 := httputil.DumpResponse(m.resp, false)
if err2 != nil {
return nr, err2
}
write(&nr, &err, w, buf)
}
return
}
// HttpLogger is an asynchronous HTTP request/response logger. It traces
// requests and responses headers in a "log" file in logger directory and dumps
// their bodies in files prefixed with the session identifiers.
// Close it to ensure pending items are correctly logged.
type HttpLogger struct {
path string
c chan *Meta
errch chan error
}
func NewLogger(basepath string) (*HttpLogger, error) {
f, err := os.Create(path.Join(basepath, "log"))
if err != nil {
return nil, err
}
logger := &HttpLogger{basepath, make(chan *Meta), make(chan error)}
go func() {
for m := range logger.c {
if _, err := m.WriteTo(f); err != nil {
log.Println("Can't write meta", err)
}
}
logger.errch <- f.Close()
}()
return logger, nil
}
func (logger *HttpLogger) LogResp(resp *http.Response, ctx *goproxy.ProxyCtx) {
body := path.Join(logger.path, fmt.Sprintf("%d_resp", ctx.Session))
from := ""
if ctx.UserData != nil {
from = ctx.UserData.(*transport.RoundTripDetails).TCPAddr.String()
}
if resp == nil {
resp = emptyResp
} else {
resp.Body = NewTeeReadCloser(resp.Body, NewFileStream(body))
}
logger.LogMeta(&Meta{
resp: resp,
err: ctx.Error,
t: time.Now(),
sess: ctx.Session,
from: from})
}
var emptyResp = &http.Response{}
var emptyReq = &http.Request{}
func (logger *HttpLogger) LogReq(req *http.Request, ctx *goproxy.ProxyCtx) {
body := path.Join(logger.path, fmt.Sprintf("%d_req", ctx.Session))
if req == nil {
req = emptyReq
} else {
req.Body = NewTeeReadCloser(req.Body, NewFileStream(body))
}
logger.LogMeta(&Meta{
req: req,
err: ctx.Error,
t: time.Now(),
sess: ctx.Session,
from: req.RemoteAddr})
}
func (logger *HttpLogger) LogMeta(m *Meta) {
logger.c <- m
}
func (logger *HttpLogger) Close() error {
close(logger.c)
return <-logger.errch
}
// TeeReadCloser extends io.TeeReader by allowing reader and writer to be
// closed.
type TeeReadCloser struct {
r io.Reader
w io.WriteCloser
c io.Closer
}
func NewTeeReadCloser(r io.ReadCloser, w io.WriteCloser) io.ReadCloser {
return &TeeReadCloser{io.TeeReader(r, w), w, r}
}
func (t *TeeReadCloser) Read(b []byte) (int, error) {
return t.r.Read(b)
}
// Close attempts to close the reader and write. It returns an error if both
// failed to Close.
func (t *TeeReadCloser) Close() error {
err1 := t.c.Close()
err2 := t.w.Close()
if err1 != nil {
return err1
}
return err2
}
// stoppableListener serves stoppableConn and tracks their lifetime to notify
// when it is safe to terminate the application.
type stoppableListener struct {
net.Listener
sync.WaitGroup
}
type stoppableConn struct {
net.Conn
wg *sync.WaitGroup
}
func newStoppableListener(l net.Listener) *stoppableListener {
return &stoppableListener{l, sync.WaitGroup{}}
}
func (sl *stoppableListener) Accept() (net.Conn, error) {
c, err := sl.Listener.Accept()
if err != nil {
return c, err
}
sl.Add(1)
return &stoppableConn{c, &sl.WaitGroup}, nil
}
func (sc *stoppableConn) Close() error {
sc.wg.Done()
return sc.Conn.Close()
}
func main() {
verbose := flag.Bool("v", false, "should every proxy request be logged to stdout")
addr := flag.String("l", ":8080", "on which address should the proxy listen")
flag.Parse()
proxy := goproxy.NewProxyHttpServer()
proxy.Verbose = *verbose
if err := os.MkdirAll("db", 0755); err != nil {
log.Fatal("Can't create dir", err)
}
logger, err := NewLogger("db")
if err != nil {
log.Fatal("can't open log file", err)
}
tr := transport.Transport{Proxy: transport.ProxyFromEnvironment}
// For every incoming request, override the RoundTripper to extract
// connection information. Store it is session context log it after
// handling the response.
proxy.OnRequest().DoFunc(func(req *http.Request, ctx *goproxy.ProxyCtx) (*http.Request, *http.Response) {
ctx.RoundTripper = goproxy.RoundTripperFunc(func(req *http.Request, ctx *goproxy.ProxyCtx) (resp *http.Response, err error) {
ctx.UserData, resp, err = tr.DetailedRoundTrip(req)
return
})
logger.LogReq(req, ctx)
return req, nil
})
proxy.OnResponse().DoFunc(func(resp *http.Response, ctx *goproxy.ProxyCtx) *http.Response {
logger.LogResp(resp, ctx)
return resp
})
l, err := net.Listen("tcp", *addr)
if err != nil {
log.Fatal("listen:", err)
}
sl := newStoppableListener(l)
ch := make(chan os.Signal)
signal.Notify(ch, os.Interrupt)
go func() {
<-ch
log.Println("Got SIGINT exiting")
sl.Add(1)
sl.Close()
logger.Close()
sl.Done()
}()
log.Println("Starting Proxy")
http.Serve(sl, proxy)
sl.Wait()
log.Println("All connections closed - exit")
}
# Content Analysis
`goproxy-jquery-version` starts an HTTP proxy on :8080. It checks HTML
responses, looks for scripts referencing jQuery library and emits warnings if
different versions of the library are being used for a given host.
Start it in one shell:
```sh
goproxy-jquery-version
```
Fetch goproxy homepage in another:
```sh
http_proxy=http://127.0.0.1:8080 wget -O - \
http://ripper234.com/p/introducing-goproxy-light-http-proxy/
```
Goproxy homepage uses jQuery and a mix of plugins. First the proxy reports the
first use of jQuery it detects for the domain. Then, because the regular
expression matching the jQuery sources is imprecise, it reports a mismatch with
a plugin reference:
```sh
2015/04/11 11:23:02 [001] WARN: ripper234.com uses //ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js
2015/04/11 11:23:02 [001] WARN: In http://ripper234.com/p/introducing-goproxy-light-http-proxy/, \
Contradicting jqueries //ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js \
http://ripper234.wpengine.netdna-cdn.com/wp-content/plugins/wp-ajax-edit-comments/js/jquery.colorbox.min.js?ver=5.0.36
```
<!doctype html>
<html>
<head>
<script src="jquery.1.4.js"></script>
</head>
<body/>
</html>
<!doctype html>
<html>
<head>
<script src="jquery.1.3.js"></script>
</head>
<body/>
</html>
package main
import (
"bytes"
"io/ioutil"
"log"
"net/http"
"net/http/httptest"
"net/url"
"strings"
"testing"
)
func equal(u, v []string) bool {
if len(u) != len(v) {
return false
}
for i, _ := range u {
if u[i] != v[i] {
return false
}
}
return true
}
func readFile(fname string, t *testing.T) string {
b, err := ioutil.ReadFile(fname)
if err != nil {
t.Fatal("readFile", err)
}
return string(b)
}
func TestDefectiveScriptParser(t *testing.T) {
if l := len(findScriptSrc(`<!DOCTYPE HTML>
<html>
<body>
<video width="320" height="240" controls="controls">
<source src="movie.mp4" type="video/mp4" />
<source src="movie.ogg" type="video/ogg" />
<source src="movie.webm" type="video/webm" />
Your browser does not support the video tag.
</video>
</body>
</html>`)); l != 0 {
t.Fail()
}
urls := findScriptSrc(readFile("w3schools.html", t))
if !equal(urls, []string{"http://partner.googleadservices.com/gampad/google_service.js",
"//translate.google.com/translate_a/element.js?cb=googleTranslateElementInit"}) {
t.Error("w3schools.html", "src scripts are not recognized", urls)
}
urls = findScriptSrc(readFile("jquery_homepage.html", t))
if !equal(urls, []string{"http://ajax.googleapis.com/ajax/libs/jquery/1.4.2/jquery.min.js",
"http://code.jquery.com/jquery-1.4.2.min.js",
"http://static.jquery.com/files/rocker/scripts/custom.js",
"http://static.jquery.com/donate/donate.js"}) {
t.Error("jquery_homepage.html", "src scripts are not recognized", urls)
}
}
func proxyWithLog() (*http.Client, *bytes.Buffer) {
proxy := NewJqueryVersionProxy()
proxyServer := httptest.NewServer(proxy)
buf := new(bytes.Buffer)
proxy.Logger = log.New(buf, "", 0)
proxyUrl, _ := url.Parse(proxyServer.URL)
tr := &http.Transport{Proxy: http.ProxyURL(proxyUrl)}
client := &http.Client{Transport: tr}
return client, buf
}
func get(t *testing.T, server *httptest.Server, client *http.Client, url string) {
resp, err := client.Get(server.URL + url)
if err != nil {
t.Fatal("cannot get proxy", err)
}
ioutil.ReadAll(resp.Body)
resp.Body.Close()
}
func TestProxyServiceTwoVersions(t *testing.T) {
var fs = httptest.NewServer(http.FileServer(http.Dir(".")))
defer fs.Close()
client, buf := proxyWithLog()
get(t, fs, client, "/w3schools.html")
get(t, fs, client, "/php_man.html")
if buf.String() != "" &&
!strings.Contains(buf.String(), " uses jquery ") {
t.Error("shouldn't warn on a single URL", buf.String())
}
get(t, fs, client, "/jquery1.html")
warnings := buf.String()
if !strings.Contains(warnings, "http://ajax.googleapis.com/ajax/libs/jquery/1.3.2/jquery.min.js") ||
!strings.Contains(warnings, "jquery.1.4.js") ||
!strings.Contains(warnings, "Contradicting") {
t.Error("contradicting jquery versions (php_man.html, w3schools.html) does not issue warning", warnings)
}
}
func TestProxyService(t *testing.T) {
var fs = httptest.NewServer(http.FileServer(http.Dir(".")))
defer fs.Close()
client, buf := proxyWithLog()
get(t, fs, client, "/jquery_homepage.html")
warnings := buf.String()
if !strings.Contains(warnings, "http://ajax.googleapis.com/ajax/libs/jquery/1.4.2/jquery.min.js") ||
!strings.Contains(warnings, "http://code.jquery.com/jquery-1.4.2.min.js") ||
!strings.Contains(warnings, "Contradicting") {
t.Error("contradicting jquery versions does not issue warning")
}
}
package main
import (
"github.com/elazarl/goproxy"
"github.com/elazarl/goproxy/ext/html"
"log"
"net/http"
"regexp"
)
var (
// who said we can't parse HTML with regexp?
scriptMatcher = regexp.MustCompile(`(?i:<script\s+)`)
srcAttrMatcher = regexp.MustCompile(`^(?i:[^>]*\ssrc=["']([^"']*)["'])`)
)
// findScripts returns all sources of HTML script tags found in input text.
func findScriptSrc(html string) []string {
srcs := make([]string, 0)
matches := scriptMatcher.FindAllStringIndex(html, -1)
for _, match := range matches {
// -1 to capture the whitespace at the end of the script tag
srcMatch := srcAttrMatcher.FindStringSubmatch(html[match[1]-1:])
if srcMatch != nil {
srcs = append(srcs, srcMatch[1])
}
}
return srcs
}
// NewJQueryVersionProxy creates a proxy checking responses HTML content, looks
// for scripts referencing jQuery library and emits warnings if different
// versions of the library are being used for a given host.
func NewJqueryVersionProxy() *goproxy.ProxyHttpServer {
proxy := goproxy.NewProxyHttpServer()
m := make(map[string]string)
jqueryMatcher := regexp.MustCompile(`(?i:jquery\.)`)
proxy.OnResponse(goproxy_html.IsHtml).Do(goproxy_html.HandleString(
func(s string, ctx *goproxy.ProxyCtx) string {
for _, src := range findScriptSrc(s) {
if !jqueryMatcher.MatchString(src) {
continue
}
prev, ok := m[ctx.Req.Host]
if ok {
if prev != src {
ctx.Warnf("In %v, Contradicting jqueries %v %v",
ctx.Req.URL, prev, src)
break
}
} else {
ctx.Warnf("%s uses jquery %s", ctx.Req.Host, src)
m[ctx.Req.Host] = src
}
}
return s
}))
return proxy
}
func main() {
proxy := NewJqueryVersionProxy()
log.Fatal(http.ListenAndServe(":8080", proxy))
}
# Request Filtering
`goproxy-no-reddit-at-work` starts an HTTP proxy on :8080. It denies requests
to "www.reddit.com" made between 8am to 5pm inclusive, local time.
Start it in one shell:
```sh
$ goproxy-no-reddit-at-work
```
Fetch reddit in another:
```sh
$ http_proxy=http://127.0.0.1:8080 wget -O - http://www.reddit.com
--2015-04-11 16:59:01-- http://www.reddit.com/
Connecting to 127.0.0.1:8080... connected.
Proxy request sent, awaiting response... 403 Forbidden
2015-04-11 16:59:01 ERROR 403: Forbidden.
```
package main
import (
"github.com/elazarl/goproxy"
"log"
"net/http"
"time"
)
func main() {
proxy := goproxy.NewProxyHttpServer()
proxy.OnRequest(goproxy.DstHostIs("www.reddit.com")).DoFunc(
func(r *http.Request, ctx *goproxy.ProxyCtx) (*http.Request, *http.Response) {
h, _, _ := time.Now().Clock()
if h >= 8 && h <= 17 {
return r, goproxy.NewResponse(r,
goproxy.ContentTypeText, http.StatusForbidden,
"Don't waste your time!")
} else {
ctx.Warnf("clock: %d, you can waste your time...", h)
}
return r, nil
})
log.Fatalln(http.ListenAndServe(":8080", proxy))
}
package main
import (
"github.com/elazarl/goproxy"
"log"
"flag"
"net"
"net/http"
)
func main() {
verbose := flag.Bool("v", false, "should every proxy request be logged to stdout")
addr := flag.String("addr", ":8080", "proxy listen address")
flag.Parse()
proxy := goproxy.NewProxyHttpServer()
proxy.Tr.Dial = func(network, addr string) (c net.Conn, err error) {
c, err = net.Dial(network, addr)
if c, ok := c.(*net.TCPConn); err != nil && ok {
c.SetKeepAlive(true)
}
return
}
proxy.Verbose = *verbose
log.Fatal(http.ListenAndServe(*addr, proxy))
}
package main
import (
"github.com/elazarl/goproxy"
"log"
"flag"
"net/http"
)
func main() {
verbose := flag.Bool("v", false, "should every proxy request be logged to stdout")
addr := flag.String("addr", ":8080", "proxy listen address")
flag.Parse()
proxy := goproxy.NewProxyHttpServer()
proxy.OnRequest().HandleConnect(goproxy.AlwaysMitm)
proxy.OnRequest().DoFunc(func (req *http.Request, ctx *goproxy.ProxyCtx) (*http.Request, *http.Response) {
if req.URL.Scheme == "https" {
req.URL.Scheme = "http"
}
return req, nil
})
proxy.Verbose = *verbose
log.Fatal(http.ListenAndServe(*addr, proxy))
}
# Gather Browsing Statistics
`goproxy-stats` starts an HTTP proxy on :8080, counts the bytes received for
web resources and prints the cumulative sum per URL every 20 seconds.
Start it in one shell:
```sh
goproxy-stats
```
Fetch goproxy homepage in another:
```sh
mkdir tmp
cd tmp
http_proxy=http://127.0.0.1:8080 wget -r -l 1 -H \
http://ripper234.com/p/introducing-goproxy-light-http-proxy/
```
Stop it after a moment. `goproxy-stats` should eventually print:
```sh
listening on :8080
statistics
http://www.telerik.com/fiddler -> 84335
http://msmvps.com/robots.txt -> 157
http://eli.thegreenplace.net/robots.txt -> 294
http://www.phdcomics.com/robots.txt -> 211
http://resharper.blogspot.com/robots.txt -> 221
http://idanz.blogli.co.il/robots.txt -> 271
http://ripper234.com/p/introducing-goproxy-light-http-proxy/ -> 44407
http://live.gnome.org/robots.txt -> 298
http://ponetium.wordpress.com/robots.txt -> 178
http://pilaheleg.blogli.co.il/robots.txt -> 321
http://pilaheleg.wordpress.com/robots.txt -> 178
http://blogli.co.il/ -> 9165
http://nimrod-code.org/robots.txt -> 289
http://www.joelonsoftware.com/robots.txt -> 1245
http://top-performance.blogspot.com/robots.txt -> 227
http://ooc-lang.org/robots.txt -> 345
http://blogs.jetbrains.com/robots.txt -> 293
```
package main
import (
"fmt"
"github.com/elazarl/goproxy"
"github.com/elazarl/goproxy/ext/html"
"io"
"log"
. "net/http"
"time"
)
type Count struct {
Id string
Count int64
}
type CountReadCloser struct {
Id string
R io.ReadCloser
ch chan<- Count
nr int64
}
func (c *CountReadCloser) Read(b []byte) (n int, err error) {
n, err = c.R.Read(b)
c.nr += int64(n)
return
}
func (c CountReadCloser) Close() error {
c.ch <- Count{c.Id, c.nr}
return c.R.Close()
}
func main() {
proxy := goproxy.NewProxyHttpServer()
timer := make(chan bool)
ch := make(chan Count, 10)
go func() {
for {
time.Sleep(20 * time.Second)
timer <- true
}
}()
go func() {
m := make(map[string]int64)
for {
select {
case c := <-ch:
m[c.Id] = m[c.Id] + c.Count
case <-timer:
fmt.Printf("statistics\n")
for k, v := range m {
fmt.Printf("%s -> %d\n", k, v)
}
}
}
}()
// IsWebRelatedText filters on html/javascript/css resources
proxy.OnResponse(goproxy_html.IsWebRelatedText).DoFunc(func(resp *Response, ctx *goproxy.ProxyCtx) *Response {
resp.Body = &CountReadCloser{ctx.Req.URL.String(), resp.Body, ch, 0}
return resp
})
fmt.Printf("listening on :8080\n")
log.Fatal(ListenAndServe(":8080", proxy))
}
# Transparent Proxy
This transparent example in goproxy is meant to show how to transparenty proxy and hijack all http and https connections while doing a man-in-the-middle to the TLS session. It requires that goproxy sees all the packets traversing out to the internet. Linux iptables rules deal with changing the source/destination IPs to act transparently, but you do need to setup your network configuration so that goproxy is a mandatory stop on the outgoing route. Primarily you can do this by placing the proxy inline. goproxy does not have any WCCP support itself; patches welcome.
## Why not explicit?
Transparent proxies are more difficult to maintain and setup from a server side, but they require no configuration on the client(s) which could be in unmanaged systems or systems that don't support a proxy configuration. See the [eavesdropper example](https://github.com/elazarl/goproxy/blob/master/examples/goproxy-eavesdropper/main.go) if you want to see an explicit proxy example.
## Potential Issues
Support for very old clients using HTTPS will fail. Clients need to send the SNI value in the TLS ClientHello which most modern clients do these days, but old clients will break.
If you're routing table allows for it, an explicit http request to goproxy will cause it to fail in an endless loop since it will try to request resources from itself repeatedly. This could be solved in the goproxy code by looking up the hostnames, but it adds a delay that is much easier/faster to handle on the routing side.
## Routing Rules
Example routing rules are included in [proxy.sh](https://github.com/elazarl/goproxy/blob/master/examples/goproxy-transparent/proxy.sh) but are best when setup using your distribution's configuration.
#!/bin/sh
# goproxy IP
GOPROXY_SERVER="10.10.10.1"
# goproxy port
GOPROXY_PORT="3129"
GOPROXY_PORT_TLS="3128"
# DO NOT MODIFY BELOW
# Load IPTABLES modules for NAT and IP conntrack support
modprobe ip_conntrack
modprobe ip_conntrack_ftp
echo 1 > /proc/sys/net/ipv4/ip_forward
echo 2 > /proc/sys/net/ipv4/conf/all/rp_filter
# Clean old firewall
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
# Write new rules
iptables -t nat -A PREROUTING -s $GOPROXY_SERVER -p tcp --dport $GOPROXY_PORT -j ACCEPT
iptables -t nat -A PREROUTING -s $GOPROXY_SERVER -p tcp --dport $GOPROXY_PORT_TLS -j ACCEPT
iptables -t nat -A PREROUTING -p tcp --dport 80 -j DNAT --to-destination $GOPROXY_SERVER:$GOPROXY_PORT
iptables -t nat -A PREROUTING -p tcp --dport 443 -j DNAT --to-destination $GOPROXY_SERVER:$GOPROXY_PORT_TLS
# The following line supports using goproxy as an explicit proxy in addition
iptables -t nat -A PREROUTING -p tcp --dport 8080 -j DNAT --to-destination $GOPROXY_SERVER:$GOPROXY_PORT
iptables -t nat -A POSTROUTING -j MASQUERADE
iptables -t mangle -A PREROUTING -p tcp --dport $GOPROXY_PORT -j DROP
iptables -t mangle -A PREROUTING -p tcp --dport $GOPROXY_PORT_TLS -j DROP
package main
import (
"bufio"
"bytes"
"flag"
"fmt"
"log"
"net"
"net/http"
"net/url"
"regexp"
"github.com/elazarl/goproxy"
"github.com/inconshreveable/go-vhost"
)
func orPanic(err error) {
if err != nil {
panic(err)
}
}
func main() {
verbose := flag.Bool("v", true, "should every proxy request be logged to stdout")
http_addr := flag.String("httpaddr", ":3129", "proxy http listen address")
https_addr := flag.String("httpsaddr", ":3128", "proxy https listen address")
flag.Parse()
proxy := goproxy.NewProxyHttpServer()
proxy.Verbose = *verbose
if proxy.Verbose {
log.Printf("Server starting up! - configured to listen on http interface %s and https interface %s", *http_addr, *https_addr)
}
proxy.NonproxyHandler = http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
if req.Host == "" {
fmt.Fprintln(w, "Cannot handle requests without Host header, e.g., HTTP 1.0")
return
}
req.URL.Scheme = "http"
req.URL.Host = req.Host
proxy.ServeHTTP(w, req)
})
proxy.OnRequest(goproxy.ReqHostMatches(regexp.MustCompile("^.*$"))).
HandleConnect(goproxy.AlwaysMitm)
proxy.OnRequest(goproxy.ReqHostMatches(regexp.MustCompile("^.*:80$"))).
HijackConnect(func(req *http.Request, client net.Conn, ctx *goproxy.ProxyCtx) {
defer func() {
if e := recover(); e != nil {
ctx.Logf("error connecting to remote: %v", e)
client.Write([]byte("HTTP/1.1 500 Cannot reach destination\r\n\r\n"))
}
client.Close()
}()
clientBuf := bufio.NewReadWriter(bufio.NewReader(client), bufio.NewWriter(client))
remote, err := connectDial(proxy, "tcp", req.URL.Host)
orPanic(err)
remoteBuf := bufio.NewReadWriter(bufio.NewReader(remote), bufio.NewWriter(remote))
for {
req, err := http.ReadRequest(clientBuf.Reader)
orPanic(err)
orPanic(req.Write(remoteBuf))
orPanic(remoteBuf.Flush())
resp, err := http.ReadResponse(remoteBuf.Reader, req)
orPanic(err)
orPanic(resp.Write(clientBuf.Writer))
orPanic(clientBuf.Flush())
}
})
go func() {
log.Fatalln(http.ListenAndServe(*http_addr, proxy))
}()
// listen to the TLS ClientHello but make it a CONNECT request instead
ln, err := net.Listen("tcp", *https_addr)
if err != nil {
log.Fatalf("Error listening for https connections - %v", err)
}
for {
c, err := ln.Accept()
if err != nil {
log.Printf("Error accepting new connection - %v", err)
continue
}
go func(c net.Conn) {
tlsConn, err := vhost.TLS(c)
if err != nil {
log.Printf("Error accepting new connection - %v", err)
}
if tlsConn.Host() == "" {
log.Printf("Cannot support non-SNI enabled clients")
return
}
connectReq := &http.Request{
Method: "CONNECT",
URL: &url.URL{
Opaque: tlsConn.Host(),
Host: net.JoinHostPort(tlsConn.Host(), "443"),
},
Host: tlsConn.Host(),
Header: make(http.Header),
}
resp := dumbResponseWriter{tlsConn}
proxy.ServeHTTP(resp, connectReq)
}(c)
}
}
// copied/converted from https.go
func dial(proxy *goproxy.ProxyHttpServer, network, addr string) (c net.Conn, err error) {
if proxy.Tr.Dial != nil {
return proxy.Tr.Dial(network, addr)
}
return net.Dial(network, addr)
}
// copied/converted from https.go
func connectDial(proxy *goproxy.ProxyHttpServer, network, addr string) (c net.Conn, err error) {
if proxy.ConnectDial == nil {
return dial(proxy, network, addr)
}
return proxy.ConnectDial(network, addr)
}
type dumbResponseWriter struct {
net.Conn
}
func (dumb dumbResponseWriter) Header() http.Header {
panic("Header() should not be called on this ResponseWriter")
}
func (dumb dumbResponseWriter) Write(buf []byte) (int, error) {
if bytes.Equal(buf, []byte("HTTP/1.0 200 OK\r\n\r\n")) {
return len(buf), nil // throw away the HTTP OK response from the faux CONNECT request
}
return dumb.Conn.Write(buf)
}
func (dumb dumbResponseWriter) WriteHeader(code int) {
panic("WriteHeader() should not be called on this ResponseWriter")
}
func (dumb dumbResponseWriter) Hijack() (net.Conn, *bufio.ReadWriter, error) {
return dumb, bufio.NewReadWriter(bufio.NewReader(dumb), bufio.NewWriter(dumb)), nil
}
package main
import (
"github.com/elazarl/goproxy"
"github.com/elazarl/goproxy/ext/image"
"image"
"log"
"net/http"
)
func main() {
proxy := goproxy.NewProxyHttpServer()
proxy.OnResponse().Do(goproxy_image.HandleImage(func(img image.Image, ctx *goproxy.ProxyCtx) image.Image {
dx, dy := img.Bounds().Dx(), img.Bounds().Dy()
nimg := image.NewRGBA(img.Bounds())
for i := 0; i < dx; i++ {
for j := 0; j <= dy; j++ {
nimg.Set(i, j, img.At(i, dy-j-1))
}
}
return nimg
}))
proxy.Verbose = true
log.Fatal(http.ListenAndServe(":8080", proxy))
}
// This example would minify standalone Javascript files (identified by their content type)
// using the command line utility YUI compressor http://yui.github.io/yuicompressor/
// Example usage:
//
// ./yui -java /usr/local/bin/java -yuicompressor ~/Downloads/yuicompressor-2.4.8.jar
// $ curl -vx localhost:8080 http://golang.org/lib/godoc/godocs.js
// (function(){function g(){var u=$("#search");if(u.length===0){return}function t(){if(....
// $ curl http://golang.org/lib/godoc/godocs.js | head -n 3
// // Copyright 2012 The Go Authors. All rights reserved.
// // Use of this source code is governed by a BSD-style
// // license that can be found in the LICENSE file.
package main
import (
"flag"
"io"
"io/ioutil"
"log"
"net/http"
"os"
"os/exec"
"path"
"strings"
"github.com/elazarl/goproxy"
)
func main() {
verbose := flag.Bool("v", false, "should every proxy request be logged to stdout")
addr := flag.String("addr", ":8080", "proxy listen address")
java := flag.String("javapath", "java", "where the Java executable is located")
yuicompressor := flag.String("yuicompressor", "", "where the yuicompressor is located, assumed to be in CWD")
yuicompressordir := flag.String("yuicompressordir", ".", "a folder to search yuicompressor in, will be ignored if yuicompressor is set")
flag.Parse()
if *yuicompressor == "" {
files, err := ioutil.ReadDir(*yuicompressordir)
if err != nil {
log.Fatal("Cannot find yuicompressor jar")
}
for _, file := range files {
if strings.HasPrefix(file.Name(), "yuicompressor") && strings.HasSuffix(file.Name(), ".jar") {
c := path.Join(*yuicompressordir, file.Name())
yuicompressor = &c
break
}
}
}
if *yuicompressor == "" {
log.Fatal("Can't find yuicompressor jar, searched yuicompressor*.jar in dir ", *yuicompressordir)
}
if _, err := os.Stat(*yuicompressor); os.IsNotExist(err) {
log.Fatal("Can't find yuicompressor jar specified ", *yuicompressor)
}
proxy := goproxy.NewProxyHttpServer()
proxy.Verbose = *verbose
proxy.OnResponse().DoFunc(func(resp *http.Response, ctx *goproxy.ProxyCtx) *http.Response {
contentType := resp.Header.Get("Content-Type")
if contentType == "application/javascript" || contentType == "application/x-javascript" {
// in real code, response should be streamed as well
var err error
cmd := exec.Command(*java, "-jar", *yuicompressor, "--type", "js")
cmd.Stdin = resp.Body
resp.Body, err = cmd.StdoutPipe()
if err != nil {
ctx.Warnf("Cannot minify content in %v: %v", ctx.Req.URL, err)
return goproxy.TextResponse(ctx.Req, "Error getting stdout pipe")
}
stderr, err := cmd.StderrPipe()
if err != nil {
ctx.Logf("Error obtaining stderr from yuicompress: %s", err)
return goproxy.TextResponse(ctx.Req, "Error getting stderr pipe")
}
if err := cmd.Start(); err != nil {
ctx.Warnf("Cannot minify content in %v: %v", ctx.Req.URL, err)
}
go func() {
defer stderr.Close()
const kb = 1024
msg, err := ioutil.ReadAll(&io.LimitedReader{stderr, 50 * kb})
if len(msg) != 0 {
ctx.Logf("Error executing yuicompress: %s", string(msg))
}
if err != nil {
ctx.Logf("Error reading stderr from yuicompress: %s", string(msg))
}
}()
}
return resp
})
log.Fatal(http.ListenAndServe(*addr, proxy))
}
package auth
import (
"bytes"
"encoding/base64"
"io/ioutil"
"net/http"
"strings"
"github.com/elazarl/goproxy"
)
var unauthorizedMsg = []byte("407 Proxy Authentication Required")
func BasicUnauthorized(req *http.Request, realm string) *http.Response {
// TODO(elazar): verify realm is well formed
return &http.Response{
StatusCode: 407,
ProtoMajor: 1,
ProtoMinor: 1,
Request: req,
Header: http.Header{"Proxy-Authenticate": []string{"Basic realm=" + realm}},
Body: ioutil.NopCloser(bytes.NewBuffer(unauthorizedMsg)),
ContentLength: int64(len(unauthorizedMsg)),
}
}
var proxyAuthorizatonHeader = "Proxy-Authorization"
func auth(req *http.Request, f func(user, passwd string) bool) bool {
authheader := strings.SplitN(req.Header.Get(proxyAuthorizatonHeader), " ", 2)
req.Header.Del(proxyAuthorizatonHeader)
if len(authheader) != 2 || authheader[0] != "Basic" {
return false
}
userpassraw, err := base64.StdEncoding.DecodeString(authheader[1])
if err != nil {
return false
}
userpass := strings.SplitN(string(userpassraw), ":", 2)
if len(userpass) != 2 {
return false
}
return f(userpass[0], userpass[1])
}
// Basic returns a basic HTTP authentication handler for requests
//
// You probably want to use auth.ProxyBasic(proxy) to enable authentication for all proxy activities
func Basic(realm string, f func(user, passwd string) bool) goproxy.ReqHandler {
return goproxy.FuncReqHandler(func(req *http.Request, ctx *goproxy.ProxyCtx) (*http.Request, *http.Response) {
if !auth(req, f) {
return nil, BasicUnauthorized(req, realm)
}
return req, nil
})
}
// BasicConnect returns a basic HTTP authentication handler for CONNECT requests
//
// You probably want to use auth.ProxyBasic(proxy) to enable authentication for all proxy activities
func BasicConnect(realm string, f func(user, passwd string) bool) goproxy.HttpsHandler {
return goproxy.FuncHttpsHandler(func(host string, ctx *goproxy.ProxyCtx) (*goproxy.ConnectAction, string) {
if !auth(ctx.Req, f) {
ctx.Resp = BasicUnauthorized(ctx.Req, realm)
return goproxy.RejectConnect, host
}
return goproxy.OkConnect, host
})
}
// ProxyBasic will force HTTP authentication before any request to the proxy is processed
func ProxyBasic(proxy *goproxy.ProxyHttpServer, realm string, f func(user, passwd string) bool) {
proxy.OnRequest().Do(Basic(realm, f))
proxy.OnRequest().HandleConnect(BasicConnect(realm, f))
}
package auth_test
import (
"encoding/base64"
"io"
"io/ioutil"
"net"
"net/http"
"net/http/httptest"
"net/url"
"os"
"os/exec"
"os/signal"
"sync/atomic"
"testing"
"github.com/elazarl/goproxy"
"github.com/elazarl/goproxy/ext/auth"
)
type ConstantHanlder string
func (h ConstantHanlder) ServeHTTP(w http.ResponseWriter, r *http.Request) {
io.WriteString(w, string(h))
}
func oneShotProxy(proxy *goproxy.ProxyHttpServer) (client *http.Client, s *httptest.Server) {
s = httptest.NewServer(proxy)
proxyUrl, _ := url.Parse(s.URL)
tr := &http.Transport{Proxy: http.ProxyURL(proxyUrl)}
client = &http.Client{Transport: tr}
return
}
func times(n int, s string) string {
r := make([]byte, 0, n*len(s))
for i := 0; i < n; i++ {
r = append(r, s...)
}
return string(r)
}
func TestBasicConnectAuthWithCurl(t *testing.T) {
expected := ":c>"
background := httptest.NewTLSServer(ConstantHanlder(expected))
defer background.Close()
proxy := goproxy.NewProxyHttpServer()
proxy.OnRequest().HandleConnect(auth.BasicConnect("my_realm", func(user, passwd string) bool {
return user == "user" && passwd == "open sesame"
}))
_, proxyserver := oneShotProxy(proxy)
defer proxyserver.Close()
cmd := exec.Command("curl",
"--silent", "--show-error", "--insecure",
"-x", proxyserver.URL,
"-U", "user:open sesame",
"-p",
"--url", background.URL+"/[1-3]",
)
out, err := cmd.CombinedOutput() // if curl got error, it'll show up in stderr
if err != nil {
t.Fatal(err, string(out))
}
finalexpected := times(3, expected)
if string(out) != finalexpected {
t.Error("Expected", finalexpected, "got", string(out))
}
}
func TestBasicAuthWithCurl(t *testing.T) {
expected := ":c>"
background := httptest.NewServer(ConstantHanlder(expected))
defer background.Close()
proxy := goproxy.NewProxyHttpServer()
proxy.OnRequest().Do(auth.Basic("my_realm", func(user, passwd string) bool {
return user == "user" && passwd == "open sesame"
}))
_, proxyserver := oneShotProxy(proxy)
defer proxyserver.Close()
cmd := exec.Command("curl",
"--silent", "--show-error",
"-x", proxyserver.URL,
"-U", "user:open sesame",
"--url", background.URL+"/[1-3]",
)
out, err := cmd.CombinedOutput() // if curl got error, it'll show up in stderr
if err != nil {
t.Fatal(err, string(out))
}
finalexpected := times(3, expected)
if string(out) != finalexpected {
t.Error("Expected", finalexpected, "got", string(out))
}
}
func TestBasicAuth(t *testing.T) {
expected := "hello"
background := httptest.NewServer(ConstantHanlder(expected))
defer background.Close()
proxy := goproxy.NewProxyHttpServer()
proxy.OnRequest().Do(auth.Basic("my_realm", func(user, passwd string) bool {
return user == "user" && passwd == "open sesame"
}))
client, proxyserver := oneShotProxy(proxy)
defer proxyserver.Close()
// without auth
resp, err := client.Get(background.URL)
if err != nil {
t.Fatal(err)
}
if resp.Header.Get("Proxy-Authenticate") != "Basic realm=my_realm" {
t.Error("Expected Proxy-Authenticate header got", resp.Header.Get("Proxy-Authenticate"))
}
if resp.StatusCode != 407 {
t.Error("Expected status 407 Proxy Authentication Required, got", resp.Status)
}
// with auth
req, err := http.NewRequest("GET", background.URL, nil)
if err != nil {
t.Fatal(err)
}
req.Header.Set("Proxy-Authorization",
"Basic "+base64.StdEncoding.EncodeToString([]byte("user:open sesame")))
resp, err = client.Do(req)
if err != nil {
t.Fatal(err)
}
if resp.StatusCode != 200 {
t.Error("Expected status 200 OK, got", resp.Status)
}
msg, err := ioutil.ReadAll(resp.Body)
if err != nil {
t.Fatal(err)
}
if string(msg) != "hello" {
t.Errorf("Expected '%s', actual '%s'", expected, string(msg))
}
}
func TestWithBrowser(t *testing.T) {
// an easy way to check if auth works with webserver
// to test, run with
// $ go test -run TestWithBrowser -- server
// configure a browser to use the printed proxy address, use the proxy
// and exit with Ctrl-C. It will throw error if your haven't acutally used the proxy
if os.Args[len(os.Args)-1] != "server" {
return
}
proxy := goproxy.NewProxyHttpServer()
println("proxy localhost port 8082")
access := int32(0)
proxy.OnRequest().Do(auth.Basic("my_realm", func(user, passwd string) bool {
atomic.AddInt32(&access, 1)
return user == "user" && passwd == "1234"
}))
l, err := net.Listen("tcp", "localhost:8082")
if err != nil {
t.Fatal(err)
}
ch := make(chan os.Signal)
signal.Notify(ch, os.Interrupt)
go func() {
<-ch
l.Close()
}()
http.Serve(l, proxy)
if access <= 0 {
t.Error("No one accessed the proxy")
}
}
// extension to goproxy that will allow you to easily filter web browser related content.
package goproxy_html
import (
"bytes"
"errors"
"io"
"io/ioutil"
"net/http"
"strings"
"code.google.com/p/go-charset/charset"
_ "code.google.com/p/go-charset/data"
"github.com/elazarl/goproxy"
)
var IsHtml goproxy.RespCondition = goproxy.ContentTypeIs("text/html")
var IsCss goproxy.RespCondition = goproxy.ContentTypeIs("text/css")
var IsJavaScript goproxy.RespCondition = goproxy.ContentTypeIs("text/javascript",
"application/javascript")
var IsJson goproxy.RespCondition = goproxy.ContentTypeIs("text/json")
var IsXml goproxy.RespCondition = goproxy.ContentTypeIs("text/xml")
var IsWebRelatedText goproxy.RespCondition = goproxy.ContentTypeIs("text/html",
"text/css",
"text/javascript", "application/javascript",
"text/xml",
"text/json")
// HandleString will receive a function that filters a string, and will convert the
// request body to a utf8 string, according to the charset specified in the Content-Type
// header.
// guessing Html charset encoding from the <META> tags is not yet implemented.
func HandleString(f func(s string, ctx *goproxy.ProxyCtx) string) goproxy.RespHandler {
return HandleStringReader(func(r io.Reader, ctx *goproxy.ProxyCtx) io.Reader {
b, err := ioutil.ReadAll(r)
if err != nil {
ctx.Warnf("Cannot read string from resp body: %v", err)
return r
}
return bytes.NewBufferString(f(string(b), ctx))
})
}
// Will receive an input stream which would convert the response to utf-8
// The given function must close the reader r, in order to close the response body.
func HandleStringReader(f func(r io.Reader, ctx *goproxy.ProxyCtx) io.Reader) goproxy.RespHandler {
return goproxy.FuncRespHandler(func(resp *http.Response, ctx *goproxy.ProxyCtx) *http.Response {
if ctx.Error != nil {
return nil
}
charsetName := ctx.Charset()
if charsetName == "" {
charsetName = "utf-8"
}
if strings.ToLower(charsetName) != "utf-8" {
r, err := charset.NewReader(charsetName, resp.Body)
if err != nil {
ctx.Warnf("Cannot convert from %v to utf-8: %v", charsetName, err)
return resp
}
tr, err := charset.TranslatorTo(charsetName)
if err != nil {
ctx.Warnf("Can't translate to %v from utf-8: %v", charsetName, err)
return resp
}
if err != nil {
ctx.Warnf("Cannot translate to %v: %v", charsetName, err)
return resp
}
newr := charset.NewTranslatingReader(f(r, ctx), tr)
resp.Body = &readFirstCloseBoth{ioutil.NopCloser(newr), resp.Body}
} else {
//no translation is needed, already at utf-8
resp.Body = &readFirstCloseBoth{ioutil.NopCloser(f(resp.Body, ctx)), resp.Body}
}
return resp
})
}
type readFirstCloseBoth struct {
r io.ReadCloser
c io.Closer
}
func (rfcb *readFirstCloseBoth) Read(b []byte) (nr int, err error) {
return rfcb.r.Read(b)
}
func (rfcb *readFirstCloseBoth) Close() error {
err1 := rfcb.r.Close()
err2 := rfcb.c.Close()
if err1 != nil && err2 != nil {
return errors.New(err1.Error() + ", " + err2.Error())
}
if err1 != nil {
return err1
}
return err2
}
package goproxy_html_test
import (
"github.com/elazarl/goproxy"
"github.com/elazarl/goproxy/ext/html"
"io/ioutil"
"net/http"
"net/http/httptest"
"net/url"
"testing"
)
type ConstantServer int
func (s ConstantServer) ServeHTTP(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/plain; charset=iso-8859-8")
//w.Header().Set("Content-Type","text/plain; charset=cp-1255")
w.Write([]byte{0xe3, 0xf3})
}
func TestCharset(t *testing.T) {
s := httptest.NewServer(ConstantServer(1))
defer s.Close()
ch := make(chan string, 2)
proxy := goproxy.NewProxyHttpServer()
proxy.OnResponse().Do(goproxy_html.HandleString(
func(s string, ctx *goproxy.ProxyCtx) string {
ch <- s
return s
}))
proxyServer := httptest.NewServer(proxy)
defer proxyServer.Close()
proxyUrl, _ := url.Parse(proxyServer.URL)
client := &http.Client{Transport: &http.Transport{Proxy: http.ProxyURL(proxyUrl)}}
resp, err := client.Get(s.URL + "/cp1255.txt")
if err != nil {
t.Fatal("GET:", err)
}
b, err := ioutil.ReadAll(resp.Body)
if err != nil {
t.Fatal("readAll:", err)
}
resp.Body.Close()
inHandleString := ""
select {
case inHandleString = <-ch:
default:
}
if len(b) != 2 || b[0] != 0xe3 || b[1] != 0xf3 {
t.Error("Did not translate back to 0xe3,0xf3, instead", b)
}
if inHandleString != "דף" {
t.Error("HandleString did not convert DALET & PEH SOFIT (דף) from ISO-8859-8 to utf-8, got", []byte(inHandleString))
}
}
package goproxy_image
import (
"bytes"
"image"
_ "image/gif"
"image/jpeg"
"image/png"
"io/ioutil"
"net/http"
. "github.com/elazarl/goproxy"
"github.com/elazarl/goproxy/regretable"
)
var RespIsImage = ContentTypeIs("image/gif",
"image/jpeg",
"image/pjpeg",
"application/octet-stream",
"image/png")
// "image/tiff" tiff support is in external package, and rarely used, so we omitted it
func HandleImage(f func(img image.Image, ctx *ProxyCtx) image.Image) RespHandler {
return FuncRespHandler(func(resp *http.Response, ctx *ProxyCtx) *http.Response {
if !RespIsImage.HandleResp(resp, ctx) {
return resp
}
if resp.StatusCode != 200 {
// we might get 304 - not modified response without data
return resp
}
contentType := resp.Header.Get("Content-Type")
const kb = 1024
regret := regretable.NewRegretableReaderCloserSize(resp.Body, 16*kb)
resp.Body = regret
img, imgType, err := image.Decode(resp.Body)
if err != nil {
regret.Regret()
ctx.Warnf("%s: %s", ctx.Req.Method+" "+ctx.Req.URL.String()+" Image from "+ctx.Req.RequestURI+"content type"+
contentType+"cannot be decoded returning original image", err)
return resp
}
result := f(img, ctx)
buf := bytes.NewBuffer([]byte{})
switch contentType {
// No gif image encoder in go - convert to png
case "image/gif", "image/png":
if err := png.Encode(buf, result); err != nil {
ctx.Warnf("Cannot encode image, returning orig %v %v", ctx.Req.URL.String(), err)
return resp
}
resp.Header.Set("Content-Type", "image/png")
case "image/jpeg", "image/pjpeg":
if err := jpeg.Encode(buf, result, nil); err != nil {
ctx.Warnf("Cannot encode image, returning orig %v %v", ctx.Req.URL.String(), err)
return resp
}
case "application/octet-stream":
switch imgType {
case "jpeg":
if err := jpeg.Encode(buf, result, nil); err != nil {
ctx.Warnf("Cannot encode image as jpeg, returning orig %v %v", ctx.Req.URL.String(), err)
return resp
}
case "png", "gif":
if err := png.Encode(buf, result); err != nil {
ctx.Warnf("Cannot encode image as png, returning orig %v %v", ctx.Req.URL.String(), err)
return resp
}
}
default:
panic("unhandlable type" + contentType)
}
resp.Body = ioutil.NopCloser(buf)
return resp
})
}
-----BEGIN RSA PRIVATE KEY-----
MIICXQIBAAKBgQC/P0FsJomPGzvdO9yreV4/faEAZ6tDVGC+VnrxnidmahUd+X7Y
2v+bR2Zb4Z05+lNyz8rN8mNgav/zjHnbh+K5HwZ1nQc61cnPIXmx6hadsEi7KvU9
sSmBGEZAyqo5S6NgTF4tt80c8ignxdnVXPK/djGNuaNYD5L+4570da0NswIDAQAB
AoGBALzIv1b4D7ARTR3NOr6V9wArjiOtMjUrdLhO+9vIp9IEA8ZsA9gjDlCEwbkP
VDnoLjnWfraff5Os6+3JjHy1fYpUiCdnk2XA6iJSL1XWKQZPt3wOunxP4lalDgED
QTRReFbA/y/Z4kSfTXpVj68ytcvSRW/N7q5/qRtbN9804jpBAkEA0s6lvH2btSLA
mcEdwhs7zAslLbdld7rvfUeP82gPPk0S6yUqTNyikqshM9AwAktHY7WvYdKl+ghZ
HTxKVC4DoQJBAOg/IAW5RbXknP+Lf7AVtBgw3E+Yfa3mcdLySe8hjxxyZq825Zmu
Rt5Qj4Lw6ifSFNy4kiiSpE/ZCukYvUXGENMCQFkPxSWlS6tzSzuqQxBGwTSrYMG3
wb6b06JyIXcMd6Qym9OMmBpw/J5KfnSNeDr/4uFVWQtTG5xO+pdHaX+3EQECQQDl
qcbY4iX1gWVfr2tNjajSYz751yoxVbkpiT9joiQLVXYFvpu+JYEfRzsjmWl0h2Lq
AftG8/xYmaEYcMZ6wSrRAkBUwiom98/8wZVlB6qbwhU1EKDFANvICGSWMIhPx3v7
MJqTIj4uJhte2/uyVvZ6DC6noWYgy+kLgqG0S97tUEG8
-----END RSA PRIVATE KEY-----
package goproxy
import (
"bufio"
"io"
"log"
"net"
"net/http"
"os"
"regexp"
"sync/atomic"
)
// The basic proxy type. Implements http.Handler.
type ProxyHttpServer struct {
// session variable must be aligned in i386
// see http://golang.org/src/pkg/sync/atomic/doc.go#L41
sess int64
// setting Verbose to true will log information on each request sent to the proxy
Verbose bool
Logger *log.Logger
NonproxyHandler http.Handler
reqHandlers []ReqHandler
respHandlers []RespHandler
httpsHandlers []HttpsHandler
Tr *http.Transport
// ConnectDial will be used to create TCP connections for CONNECT requests
// if nil Tr.Dial will be used
ConnectDial func(network string, addr string) (net.Conn, error)
}
var hasPort = regexp.MustCompile(`:\d+$`)
func copyHeaders(dst, src http.Header) {
for k, _ := range dst {
dst.Del(k)
}
for k, vs := range src {
for _, v := range vs {
dst.Add(k, v)
}
}
}
func isEof(r *bufio.Reader) bool {
_, err := r.Peek(1)
if err == io.EOF {
return true
}
return false
}
func (proxy *ProxyHttpServer) filterRequest(r *http.Request, ctx *ProxyCtx) (req *http.Request, resp *http.Response) {
req = r
for _, h := range proxy.reqHandlers {
req, resp = h.Handle(r, ctx)
// non-nil resp means the handler decided to skip sending the request
// and return canned response instead.
if resp != nil {
break
}
}
return
}
func (proxy *ProxyHttpServer) filterResponse(respOrig *http.Response, ctx *ProxyCtx) (resp *http.Response) {
resp = respOrig
for _, h := range proxy.respHandlers {
ctx.Resp = resp
resp = h.Handle(resp, ctx)
}
return
}
func removeProxyHeaders(ctx *ProxyCtx, r *http.Request) {
r.RequestURI = "" // this must be reset when serving a request with the client
ctx.Logf("Sending request %v %v", r.Method, r.URL.String())
// If no Accept-Encoding header exists, Transport will add the headers it can accept
// and would wrap the response body with the relevant reader.
r.Header.Del("Accept-Encoding")
// curl can add that, see
// http://homepage.ntlworld.com/jonathan.deboynepollard/FGA/web-proxy-connection-header.html
r.Header.Del("Proxy-Connection")
r.Header.Del("Proxy-Authenticate")
r.Header.Del("Proxy-Authorization")
// Connection, Authenticate and Authorization are single hop Header:
// http://www.w3.org/Protocols/rfc2616/rfc2616.txt
// 14.10 Connection
// The Connection general-header field allows the sender to specify
// options that are desired for that particular connection and MUST NOT
// be communicated by proxies over further connections.
r.Header.Del("Connection")
}
// Standard net/http function. Shouldn't be used directly, http.Serve will use it.
func (proxy *ProxyHttpServer) ServeHTTP(w http.ResponseWriter, r *http.Request) {
//r.Header["X-Forwarded-For"] = w.RemoteAddr()
if r.Method == "CONNECT" {
proxy.handleHttps(w, r)
} else {
ctx := &ProxyCtx{Req: r, Session: atomic.AddInt64(&proxy.sess, 1), proxy: proxy}
var err error
ctx.Logf("Got request %v %v %v %v", r.URL.Path, r.Host, r.Method, r.URL.String())
if !r.URL.IsAbs() {
proxy.NonproxyHandler.ServeHTTP(w, r)
return
}
r, resp := proxy.filterRequest(r, ctx)
if resp == nil {
removeProxyHeaders(ctx, r)
resp, err = ctx.RoundTrip(r)
if err != nil {
ctx.Error = err
resp = proxy.filterResponse(nil, ctx)
if resp == nil {
ctx.Logf("error read response %v %v:", r.URL.Host, err.Error())
http.Error(w, err.Error(), 500)
return
}
}
ctx.Logf("Received response %v", resp.Status)
}
origBody := resp.Body
resp = proxy.filterResponse(resp, ctx)
ctx.Logf("Copying response to client %v [%d]", resp.Status, resp.StatusCode)
// http.ResponseWriter will take care of filling the correct response length
// Setting it now, might impose wrong value, contradicting the actual new
// body the user returned.
// We keep the original body to remove the header only if things changed.
// This will prevent problems with HEAD requests where there's no body, yet,
// the Content-Length header should be set.
if origBody != resp.Body {
resp.Header.Del("Content-Length")
}
copyHeaders(w.Header(), resp.Header)
w.WriteHeader(resp.StatusCode)
nr, err := io.Copy(w, resp.Body)
if err := resp.Body.Close(); err != nil {
ctx.Warnf("Can't close response body %v", err)
}
ctx.Logf("Copied %v bytes to client error=%v", nr, err)
}
}
// New proxy server, logs to StdErr by default
func NewProxyHttpServer() *ProxyHttpServer {
proxy := ProxyHttpServer{
Logger: log.New(os.Stderr, "", log.LstdFlags),
reqHandlers: []ReqHandler{},
respHandlers: []RespHandler{},
httpsHandlers: []HttpsHandler{},
NonproxyHandler: http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
http.Error(w, "This is a proxy server. Does not respond to non-proxy requests.", 500)
}),
Tr: &http.Transport{TLSClientConfig: tlsClientSkipVerify,
Proxy: http.ProxyFromEnvironment},
}
proxy.ConnectDial = dialerFromEnv(&proxy)
return &proxy
}
package regretable
import (
"io"
)
// A RegretableReader will allow you to read from a reader, and then
// to "regret" reading it, and push back everything you've read.
// For example,
// rb := NewRegretableReader(bytes.NewBuffer([]byte{1,2,3}))
// var b = make([]byte,1)
// rb.Read(b) // b[0] = 1
// rb.Regret()
// ioutil.ReadAll(rb.Read) // returns []byte{1,2,3},nil
type RegretableReader struct {
reader io.Reader
overflow bool
r, w int
buf []byte
}
var defaultBufferSize = 500
// Same as RegretableReader, but allows closing the underlying reader
type RegretableReaderCloser struct {
RegretableReader
c io.Closer
}
// Closes the underlying readCloser, you cannot regret after closing the stream
func (rbc *RegretableReaderCloser) Close() error {
return rbc.c.Close()
}
// initialize a RegretableReaderCloser with underlying readCloser rc
func NewRegretableReaderCloser(rc io.ReadCloser) *RegretableReaderCloser {
return &RegretableReaderCloser{*NewRegretableReader(rc), rc}
}
// initialize a RegretableReaderCloser with underlying readCloser rc
func NewRegretableReaderCloserSize(rc io.ReadCloser, size int) *RegretableReaderCloser {
return &RegretableReaderCloser{*NewRegretableReaderSize(rc, size), rc}
}
// The next read from the RegretableReader will be as if the underlying reader
// was never read (or from the last point forget is called).
func (rb *RegretableReader) Regret() {
if rb.overflow {
panic("regretting after overflow makes no sense")
}
rb.r = 0
}
// Will "forget" everything read so far.
// rb := NewRegretableReader(bytes.NewBuffer([]byte{1,2,3}))
// var b = make([]byte,1)
// rb.Read(b) // b[0] = 1
// rb.Forget()
// rb.Read(b) // b[0] = 2
// rb.Regret()
// ioutil.ReadAll(rb.Read) // returns []byte{2,3},nil
func (rb *RegretableReader) Forget() {
if rb.overflow {
panic("forgetting after overflow makes no sense")
}
rb.r = 0
rb.w = 0
}
// initialize a RegretableReader with underlying reader r, whose buffer is size bytes long
func NewRegretableReaderSize(r io.Reader, size int) *RegretableReader {
return &RegretableReader{reader: r, buf: make([]byte, size) }
}
// initialize a RegretableReader with underlying reader r
func NewRegretableReader(r io.Reader) *RegretableReader {
return NewRegretableReaderSize(r, defaultBufferSize)
}
// reads from the underlying reader. Will buffer all input until Regret is called.
func (rb *RegretableReader) Read(p []byte) (n int, err error) {
if rb.overflow {
return rb.reader.Read(p)
}
if rb.r < rb.w {
n = copy(p, rb.buf[rb.r:rb.w])
rb.r += n
return
}
n, err = rb.reader.Read(p)
bn := copy(rb.buf[rb.w:], p[:n])
rb.w, rb.r = rb.w + bn, rb.w + n
if bn < n {
rb.overflow = true
}
return
}
package regretable_test
import (
. "github.com/elazarl/goproxy/regretable"
"bytes"
"io"
"io/ioutil"
"strings"
"testing"
)
func TestRegretableReader(t *testing.T) {
buf := new(bytes.Buffer)
mb := NewRegretableReader(buf)
word := "12345678"
buf.WriteString(word)
fivebytes := make([]byte, 5)
mb.Read(fivebytes)
mb.Regret()
s, _ := ioutil.ReadAll(mb)
if string(s) != word {
t.Errorf("Uncommited read is gone, [%d,%d] actual '%v' expected '%v'\n", len(s), len(word), string(s), word)
}
}
func TestRegretableEmptyRead(t *testing.T) {
buf := new(bytes.Buffer)
mb := NewRegretableReader(buf)
word := "12345678"
buf.WriteString(word)
zero := make([]byte, 0)
mb.Read(zero)
mb.Regret()
s, err := ioutil.ReadAll(mb)
if string(s) != word {
t.Error("Uncommited read is gone, actual:", string(s), "expected:", word, "err:", err)
}
}
func TestRegretableAlsoEmptyRead(t *testing.T) {
buf := new(bytes.Buffer)
mb := NewRegretableReader(buf)
word := "12345678"
buf.WriteString(word)
one := make([]byte, 1)
zero := make([]byte, 0)
five := make([]byte, 5)
mb.Read(one)
mb.Read(zero)
mb.Read(five)
mb.Regret()
s, _ := ioutil.ReadAll(mb)
if string(s) != word {
t.Error("Uncommited read is gone", string(s), "expected", word)
}
}
func TestRegretableRegretBeforeRead(t *testing.T) {
buf := new(bytes.Buffer)
mb := NewRegretableReader(buf)
word := "12345678"
buf.WriteString(word)
five := make([]byte, 5)
mb.Regret()
mb.Read(five)
s, err := ioutil.ReadAll(mb)
if string(s) != "678" {
t.Error("Uncommited read is gone", string(s), len(string(s)), "expected", "678", len("678"), "err:", err)
}
}
func TestRegretableFullRead(t *testing.T) {
buf := new(bytes.Buffer)
mb := NewRegretableReader(buf)
word := "12345678"
buf.WriteString(word)
twenty := make([]byte, 20)
mb.Read(twenty)
mb.Regret()
s, _ := ioutil.ReadAll(mb)
if string(s) != word {
t.Error("Uncommited read is gone", string(s), len(string(s)), "expected", word, len(word))
}
}
func assertEqual(t *testing.T, expected, actual string) {
if expected!=actual {
t.Fatal("Expected", expected, "actual", actual)
}
}
func assertReadAll(t *testing.T, r io.Reader) string {
s, err := ioutil.ReadAll(r)
if err!=nil {
t.Fatal("error when reading", err)
}
return string(s)
}
func TestRegretableRegretTwice(t *testing.T) {
buf := new(bytes.Buffer)
mb := NewRegretableReader(buf)
word := "12345678"
buf.WriteString(word)
assertEqual(t, word, assertReadAll(t, mb))
mb.Regret()
assertEqual(t, word, assertReadAll(t, mb))
mb.Regret()
assertEqual(t, word, assertReadAll(t, mb))
}
type CloseCounter struct {
r io.Reader
closed int
}
func (cc *CloseCounter) Read(b []byte) (int, error) {
return cc.r.Read(b)
}
func (cc *CloseCounter) Close() error {
cc.closed++
return nil
}
func assert(t *testing.T, b bool, msg string) {
if !b {
t.Errorf("Assertion Error: %s", msg)
}
}
func TestRegretableCloserSizeRegrets(t *testing.T) {
defer func() {
if r := recover(); r == nil || !strings.Contains(r.(string), "regret") {
t.Error("Did not panic when regretting overread buffer:", r)
}
}()
buf := new(bytes.Buffer)
buf.WriteString("123456")
mb := NewRegretableReaderCloserSize(ioutil.NopCloser(buf), 3)
mb.Read(make([]byte, 4))
mb.Regret()
}
func TestRegretableCloserRegretsClose(t *testing.T) {
buf := new(bytes.Buffer)
cc := &CloseCounter{buf, 0}
mb := NewRegretableReaderCloser(cc)
word := "12345678"
buf.WriteString(word)
mb.Read([]byte{0})
mb.Close()
if cc.closed != 1 {
t.Error("RegretableReaderCloser ignores Close")
}
mb.Regret()
mb.Close()
if cc.closed != 2 {
t.Error("RegretableReaderCloser does ignore Close after regret")
}
// TODO(elazar): return an error if client issues Close more than once after regret
}
package goproxy
import (
"bytes"
"io/ioutil"
"net/http"
)
// Will generate a valid http response to the given request the response will have
// the given contentType, and http status.
// Typical usage, refuse to process requests to local addresses:
//
// proxy.OnRequest(IsLocalHost()).DoFunc(func(r *http.Request, ctx *goproxy.ProxyCtx) (*http.Request,*http.Response) {
// return nil,NewResponse(r,goproxy.ContentTypeHtml,http.StatusUnauthorized,
// `<!doctype html><html><head><title>Can't use proxy for local addresses</title></head><body/></html>`)
// })
func NewResponse(r *http.Request, contentType string, status int, body string) *http.Response {
resp := &http.Response{}
resp.Request = r
resp.TransferEncoding = r.TransferEncoding
resp.Header = make(http.Header)
resp.Header.Add("Content-Type", contentType)
resp.StatusCode = status
buf := bytes.NewBufferString(body)
resp.ContentLength = int64(buf.Len())
resp.Body = ioutil.NopCloser(buf)
return resp
}
const (
ContentTypeText = "text/plain"
ContentTypeHtml = "text/html"
)
// Alias for NewResponse(r,ContentTypeText,http.StatusAccepted,text)
func TextResponse(r *http.Request, text string) *http.Response {
return NewResponse(r, ContentTypeText, http.StatusAccepted, text)
}
package goproxy
import (
"crypto/rsa"
"crypto/sha1"
"crypto/tls"
"crypto/x509"
"crypto/x509/pkix"
"math/big"
"net"
"runtime"
"sort"
"time"
)
func hashSorted(lst []string) []byte {
c := make([]string, len(lst))
copy(c, lst)
sort.Strings(c)
h := sha1.New()
for _, s := range c {
h.Write([]byte(s + ","))
}
return h.Sum(nil)
}
func hashSortedBigInt(lst []string) *big.Int {
rv := new(big.Int)
rv.SetBytes(hashSorted(lst))
return rv
}
var goproxySignerVersion = ":goroxy1"
func signHost(ca tls.Certificate, hosts []string) (cert tls.Certificate, err error) {
var x509ca *x509.Certificate
// Use the provided ca and not the global GoproxyCa for certificate generation.
if x509ca, err = x509.ParseCertificate(ca.Certificate[0]); err != nil {
return
}
start := time.Unix(0, 0)
end, err := time.Parse("2006-01-02", "2049-12-31")
if err != nil {
panic(err)
}
hash := hashSorted(append(hosts, goproxySignerVersion, ":"+runtime.Version()))
serial := new(big.Int)
serial.SetBytes(hash)
template := x509.Certificate{
// TODO(elazar): instead of this ugly hack, just encode the certificate and hash the binary form.
SerialNumber: serial,
Issuer: x509ca.Subject,
Subject: pkix.Name{
Organization: []string{"GoProxy untrusted MITM proxy Inc"},
},
NotBefore: start,
NotAfter: end,
KeyUsage: x509.KeyUsageKeyEncipherment | x509.KeyUsageDigitalSignature,
ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth},
BasicConstraintsValid: true,
}
for _, h := range hosts {
if ip := net.ParseIP(h); ip != nil {
template.IPAddresses = append(template.IPAddresses, ip)
} else {
template.DNSNames = append(template.DNSNames, h)
}
}
var csprng CounterEncryptorRand
if csprng, err = NewCounterEncryptorRandFromKey(ca.PrivateKey, hash); err != nil {
return
}
var certpriv *rsa.PrivateKey
if certpriv, err = rsa.GenerateKey(&csprng, 1024); err != nil {
return
}
var derBytes []byte
if derBytes, err = x509.CreateCertificate(&csprng, &template, x509ca, &certpriv.PublicKey, ca.PrivateKey); err != nil {
return
}
return tls.Certificate{
Certificate: [][]byte{derBytes, ca.Certificate[0]},
PrivateKey: certpriv,
}, nil
}
package goproxy
import (
"crypto/tls"
"crypto/x509"
"io/ioutil"
"net/http"
"net/http/httptest"
"os"
"os/exec"
"strings"
"testing"
"time"
)
func orFatal(msg string, err error, t *testing.T) {
if err != nil {
t.Fatal(msg, err)
}
}
type ConstantHanlder string
func (h ConstantHanlder) ServeHTTP(w http.ResponseWriter, r *http.Request) {
w.Write([]byte(h))
}
func getBrowser(args []string) string {
for i, arg := range args {
if arg == "-browser" && i+1 < len(arg) {
return args[i+1]
}
if strings.HasPrefix(arg, "-browser=") {
return arg[len("-browser="):]
}
}
return ""
}
func TestSingerTls(t *testing.T) {
cert, err := signHost(GoproxyCa, []string{"example.com", "1.1.1.1", "localhost"})
orFatal("singHost", err, t)
cert.Leaf, err = x509.ParseCertificate(cert.Certificate[0])
orFatal("ParseCertificate", err, t)
expected := "key verifies with Go"
server := httptest.NewUnstartedServer(ConstantHanlder(expected))
defer server.Close()
server.TLS = &tls.Config{Certificates: []tls.Certificate{cert, GoproxyCa}}
server.TLS.BuildNameToCertificate()
server.StartTLS()
certpool := x509.NewCertPool()
certpool.AddCert(GoproxyCa.Leaf)
tr := &http.Transport{
TLSClientConfig: &tls.Config{RootCAs: certpool},
}
asLocalhost := strings.Replace(server.URL, "127.0.0.1", "localhost", -1)
req, err := http.NewRequest("GET", asLocalhost, nil)
orFatal("NewRequest", err, t)
resp, err := tr.RoundTrip(req)
orFatal("RoundTrip", err, t)
txt, err := ioutil.ReadAll(resp.Body)
orFatal("ioutil.ReadAll", err, t)
if string(txt) != expected {
t.Errorf("Expected '%s' got '%s'", expected, string(txt))
}
browser := getBrowser(os.Args)
if browser != "" {
exec.Command(browser, asLocalhost).Run()
time.Sleep(10 * time.Second)
}
}
func TestSingerX509(t *testing.T) {
cert, err := signHost(GoproxyCa, []string{"example.com", "1.1.1.1", "localhost"})
orFatal("singHost", err, t)
cert.Leaf, err = x509.ParseCertificate(cert.Certificate[0])
orFatal("ParseCertificate", err, t)
certpool := x509.NewCertPool()
certpool.AddCert(GoproxyCa.Leaf)
orFatal("VerifyHostname", cert.Leaf.VerifyHostname("example.com"), t)
orFatal("CheckSignatureFrom", cert.Leaf.CheckSignatureFrom(GoproxyCa.Leaf), t)
_, err = cert.Leaf.Verify(x509.VerifyOptions{
DNSName: "example.com",
Roots: certpool,
})
orFatal("Verify", err, t)
}
package transport
import "net/http"
type RoundTripper interface {
// RoundTrip executes a single HTTP transaction, returning
// the Response for the request req. RoundTrip should not
// attempt to interpret the response. In particular,
// RoundTrip must return err == nil if it obtained a response,
// regardless of the response's HTTP status code. A non-nil
// err should be reserved for failure to obtain a response.
// Similarly, RoundTrip should not attempt to handle
// higher-level protocol details such as redirects,
// authentication, or cookies.
//
// RoundTrip should not modify the request, except for
// consuming the Body. The request's URL and Header fields
// are guaranteed to be initialized.
RoundTrip(*http.Request) (*http.Response, error)
DetailedRoundTrip(*http.Request) (*RoundTripDetails, *http.Response, error)
}
package transport
import (
"fmt"
"strings"
)
type badStringError struct {
what string
str string
}
func (e *badStringError) Error() string { return fmt.Sprintf("%s %q", e.what, e.str) }
func hasPort(s string) bool { return strings.LastIndex(s, ":") > strings.LastIndex(s, "]") }
# Copyright 2015 Google Inc. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
FROM scratch
ADD goproxy goproxy
ENTRYPOINT ["/goproxy"]
all: push
TAG = 0.1
goproxy: goproxy.go
CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -ldflags '-w' ./goproxy.go
image: goproxy
docker build -t gcr.io/google_containers/goproxy:$(TAG) .
push: image
gcloud docker push gcr.io/google_containers/goproxy:$(TAG)
clean:
rm -f goproxy
/*
Copyright 2015 The Kubernetes Authors All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package main
import (
"log"
"net/http"
"github.com/elazarl/goproxy"
)
func main() {
proxy := goproxy.NewProxyHttpServer()
proxy.Verbose = true
log.Fatal(http.ListenAndServe(":8080", proxy))
}
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment