back to portfolio

Project Sencomment, HU Berlin

Oct 2011, 12 months

Position

Python Developer

Responsibilities

Work within a lean software process to design and develop new product features
Acquiring skills enabling the creation of scalable functionality with Python
Documenting new features
Writing unit tests
Writing functional tests
Resolve errors and performance issues
Creating design specifications based on requirements for new and existing products
Share knowledge and support
Constant evaluation, optimization and enhancement of existing functionalities
Leading the backend team

Link

https://fis.hu-berlin.de/converis/portal/Project/403360821?auxfun=&lang=de_DE

Technologies

Project description

SenComment used the modern Natural Language Processing (NLP) tools, for discovering the author's social properties based on the linguistic patterns found texts.

Further this information would be used for generating narrow targeted online ads.

The system was containing of multiple components, allowing the data gathering, transformation, analysis and delivery to end ads platform

First a scrapy based web crawler collects the information from different sources and organizes them into two big groups: the text marked with gender and social properties of the author and those corresponding to the product descriptions. The texts are cleaned stored in the relational database.

At the second step the machinery analyzes the language features of gathered texts and matches them with with each other, building the distribution of typical buyer of the product: gender, age and education level.

At the third step the system generates fine grained marketing information for needed for importing it into advertising platform: target groups, product description and media.

The access to data was enabled with django based web interface.