Tag Archives: python

Adventure into ReportLab and RML

Posted on by 0 comment

My journey began after reading a long report on network statistics that one of my co-workers was required to write.
I have always been enthusiastic about not writing reports but generating them, and as much as I love manual typesetting some documents are just to vast and to complicated to manually typeset every paragraph to make sure it looks just right.
The document we will be generating needs to pull loads of data from Intrusion Prevention/Detection and other network devices we will then be manipulating the data calculating some more graphing some of it then putting it all into tables. I’ve always wanted a project to do in ReportLab. I began looking into ReportLab and officially they offer 2 versions; ReportLab Plus and ReportLab OpenSource. Unfortunately ReportLab OpenSource does not support the ReportLab Markup Language(RML) which is in essence is an XML dialect. The licence for ReportLab plus was a but out of my price range so I went searching and came up with https://pypi.python.org/pypi/z3c.rml an RML parser that comes with an rml2pdf function. But first I had to get z3c.rml and opensource ReportLab installed onto my MacBook.
I kept getting this error;
clang: error: unknown argument
I found the solution here http://bruteforce.gr/bypassing-clang-error-unknown-argument.html where you can suppress the error by prefixing your pip install command with the following
ARCHFLAGS=-Wno-error=unused-command-line-argument-hard-error-in-future pip install
This seemed to work and I was finally able to import reportlab z3c.rml and all the other dependancies.

I started off with a file simply importing the modules and calling rml2pdf.go but this evolved a bit as I incorporated templating so I came up with this .py file to read in an XML file apply mako rendering then create an RML file which I render to PDF like so.

import z3c.rml.tests
from z3c.rml import rml2pdf, attr
from mako.template import Template
mytemplate = Template(filename='btest.xml')
rml2pdf.go("btest.rml", "test.pdf")

I then proceeded to build up my sample document.
which you can find on bitbucket here https://bitbucket.org/alec_langford/rml-bits
where I set a border imported some images put in headers, footers and page numbers.

So hopefully this will be the way we typeset future reports and I won’t need to spend a whole bunch of time fixing badly processed word documents.

Category: Other code, Python, Software | Tags: , ,

Building a Quick and Dirty Url Shortener

Posted on by 0 comment

At work last week we were discussing the security implications of url shortening services, such as tinyURL, biy.ly and goo.gl not only the fact that they can be used to hide malicious URLs for use in phishing attacks but the problem’s we’re having are:

  • Users in more restrictive access groups not being able to click links from these services
  • But worse, some users are using this service to shorten intranet links

Now that second point is an issue for me; if a shortening service were hacked our server names could be leaked to the world.

The two obvious solutions were ban all users from using such services or run our own internal service

My instinct told me that one shouldn’t be to hard to build.

So Here it is in less than 50 lines

from SimpleHTTPServer import SimpleHTTPRequestHandler
import StringIO,os,BaseHTTPServer,sqlite3
if "urls.db" in os.listdir("."):
    con = sqlite3.connect("urls.db")
    con = sqlite3.connect("urls.db")
    c.execute("create table shorts (id integer primary key, url varchar unique)")
server = BaseHTTPServer.HTTPServer
server_address = ("", 8000)
class MyHandler(SimpleHTTPRequestHandler):
    def send_head(self):
        body,response = " ",200
        if self.path=="""/""":pass
        elif self.path.endswith("+"):
            c.execute('SELECT url FROM shorts WHERE id=(?)', (self.path[1:-1].decode("base64"),))
            boady = s[0]        
        elif r"/add?" not in self.path:
            c.execute('SELECT url FROM shorts WHERE id=?', (self.path[1:].decode("base64"),))
                c.execute("insert into shorts(url) values (?)", (x,))
            except sqlite3.IntegrityError:pass
            c.execute('SELECT id FROM shorts WHERE url=(?)', (x,))
            body = "ok. " + str(s[0]).encode("base64")
        self.send_header("Content-type", "text/html; charset=utf-8")  
        self.send_header("Content-Length", str(len(body)))  
        if response==301:
                return StringIO.StringIO(body)
httpd = server(server_address, MyHandler)
print "Starting server..."
except KeyboardInterrupt:
Category: Python, Software | Tags: , ,

Automajikly updating a log page with JQuery

Posted on by 0 comment

I was developing a a web application at work for use on the intranet. And if you’re anything like the security nut I am you love logging just as much as I do. I love logging so much I have a page for just about every I use generally my log pages look something like

import os
print "Content-Type:text/html"
print '<br/>'.join(os.popen("tail -100 somelog.log").read().split("n"))

Now this is ok but wouldn’t it be cool if it updated without the page refreshing?
Now I’m not very good at Jquery so I had no idea to start but eventually I came across Jeff Star’s blog post http://perishablepress.com/ajax-error-log/ which was pretty much exactly what I was after without all the fancy 404 logging since my web framework does all that.
So quite simply I took this code

		<title>Ajax Error Log</title>
		<!-- Ajax Error Log @ http://perishablepress.com/ajax-error-log/ -->
		<meta http-equiv="content-type" content="text/html; charset=UTF-8">
			pre {
				font: 10px/1.5 Courier, "Courier New", mono;
				background-color: #efefef; border: 1px solid #ccc;
				width: 700px; margin: 7px; padding: 10px;
				white-space: pre-wrap;
		<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.3.0/jquery.min.js "></script>
			$(document).ready(function() {
				var refreshId = setInterval(function() {
				}, 2000); // refresh time (default = 2000 ms = 2 seconds)
		<noscript><div id="response"><h1>JavaScript is required for this demo.</h1></div></noscript>
		<div id="results"></div>

And changed AjaxErrorLog.php to the cgi script tailing my log and presto a live log feed.

Dealing with unicode in Python

Posted on by 2 comments

I say transliterating, but I really just mean coping with strings involving unprintable characters…


Today I came across a neat little trick which I hope helps you as much as I.

take the following print statement

print "blahÆlec"

this will break

but there are some cases where exact spelling isn’t important you just to to out put the data so you can glace at it, you can actually give the str decode method a nifty argument “ignore” eg.

print "blahÆlec".decode("ascii","ignore")

and this simply drops the unprintable characters.


Category: Python | Tags: , ,