## 30 December 2009

### Some random NIPS thoughts...

I missed the first two days of NIPS due to teaching. Which is sad -- I heard there were great things on the first day. I did end up seeing a lot that was nice. But since I missed stuff, I'll instead post some paper suggests from one of my students, Piyush Rai, who was there. You can tell his biases from his selections, but that's life :). More of my thoughts after his notes...

Says Piyush:

There was an interesting tutorial by Gunnar Martinsson on using randomization to speed-up matrix factorization (SVD, PCA etc) of really really large matrices (by "large", I mean something like 106 x 106). People typically use Krylov subspace methods (e.g., the Lanczos algo) but these require multiple passes over the data. It turns out that with the randomized approach, you can do it in a single pass or a small number of passes (so it can be useful in a streaming setting). The idea is quite simple. Let's assume you want the top K evals/evecs of a large matrix A. The randomized method draws K *random* vectors from a Gaussian and uses them in some way (details here) to get a "smaller version" of A on which doing SVD can be very cheap. Having got the evals/evecs of B, a simple transformation will give you the same for the original matrix A.
The success of many matrix factorization methods (e.g., the Lanczos) also depends on how quickly the spectrum decays (eigenvalues) and they also suggest ways of dealing with cases where the spectrum doesn't quite decay that rapidly.

Some papers from the main conference that I found interesting:

Distribution Matching for Transduction (Alex Smola and 2 other guys): They use maximum mean discrepancy (MMD) to do predictions in a transduction setting (i.e., when you also have the test data at training time). The idea is to use the fact that we expect the output functions f(X) and f(X') to be the same or close to each other (X are training and X' are test inputs). So instead of using the standard regularized objective used in the inductive setting, they use the distribution discrepancy (measured by say D) of f(X) and f(X') as a regularizer. D actually decomposes over pairs of training and test examples so one can use a stochastic approximation of D (D_i for the i-th pair of training and test inputs) and do something like an SGD.

Semi-supervised Learning using Sparse Eigenfunction Bases (Sinha and Belkin from Ohio): This paper uses the cluster assumption of semi-supervised learning. They use unlabeled data to construct a set of basis functions and then use labeled data in the LASSO framework to select a sparse combination of basis functions to learn the final classifier.

Streaming k-means approximation (Nir Ailon et al.): This paper does an online optimization of the k-means objective function. The algo is based on the previously proposed kmeans++ algorithm.

The Wisdom of Crowds in the Recollection of Order Information. It's about aggregating rank information from various individuals to reconstruct the global ordering.

Dirichlet-Bernoulli Alignment: A Generative Model for Multi-Class Multi-Label Multi-Instance Corpora (by some folks at gatech): The problem setting is interesting here. Here the "multi-instance" is a bit of a misnomer. It means that each example in turn can consists of several sub-examples (which they call instances). E.g., a document consists of several paragraphs, or a webpage consists of text, images, videos.

Construction of Nonparametric Bayesian Models from Parametric Bayes Equations (Peter Orbanz): If you care about Bayesian nonparametrics. :) It basically builds on the Kolmogorov consistency theorem to formalize and sort of gives a recipe for the construction of nonparametric Bayesian models from their parametric counterparts. Seemed to be a good step in the right direction.

Indian Buffet Processes with Power-law Behavior (YWT and Dilan Gorur): This paper actually does the exact opposite of what I had thought of doing for IBP. The IBP (akin to the sense of the Dirichlet process) encourages the "rich-gets-richer" phenomena in the sense that a dish that has been already selected by a lot of customers is highly likely to be selected by future customers as well. This leads to the expected number of dishes (and thus the latent-features) to be something like O(alpha* log n). This paper tries to be even more aggressive and makes the relationship have a power-law behavior. What I wanted to do was a reverse behavior -- maybe more like a "socialist IBP" :) where the customers in IBP are sort of evenly distributed across the dishes.
The rest of this post are random thoughts that occurred to me at NIPS. Maybe some of them will get other people's wheels turning? This was originally an email I sent to my students, but I figured I might as well post it for the world. But forgive the lack of capitalization :):

persi diaconis' invited talk about reinforcing random walks... that is, you take a random walk, but every time you cross an edge, you increase the probability that you re-cross that edge (see coppersmith + diaconis, rolles + diaconis).... this relates to a post i had a while ago: nlpers.blogspot.com/2007/04/multinomial-on-graph.html ... i'm thinking that you could set up a reinforcing random walk on a graph to achieve this. the key problem is how to compute things -- basically want you want is to know for two nodes i,j in a graph and some n >= 0, whether there exists a walk from i to j that takes exactly n steps. seems like you could craft a clever data structure to answer this question, then set up a graph multinomial based on this, with reinforcement (the reinforcement basically looks like the additive counts you get from normal multinomials)... if you force n=1 and have a fully connected graph, you should recover a multinomial/dirichlet pair.

also from persi's talk, persi and some guy sergei (sergey?) have a paper on variable length markov chains that might be interesting to look at, perhaps related to frank wood's sequence memoizer paper from icml last year.

finally, also from persi's talk, steve mc_something from ohio has a paper on using common gamma distributions in different rows to set dependencies among markov chains... this is related to something i was thinking about a while ago where you want to set up transition matrices with stick-breaking processes, and to have a common, global, set of sticks that you draw from... looks like this steve mc_something guy has already done this (or something like it).

not sure what made me think of this, but related to a talk we had here a few weeks ago about unit tests in scheme, where they basically randomly sample programs to "hope" to find bugs... what about setting this up as an RL problem where your reward is high if you're able to find a bug with a "simple" program... something like 0 if you don't find a bug, or 1/`|P|` if you find a bug with program P. (i think this came up when i was talking to percy -- liang, the other one -- about some semantics stuff he's been looking at.) afaik, no one in PL land has tried ANYTHING remotely like this... it's a little tricky because of the infinite but discrete state space (of programs), but something like an NN-backed Q-learning might do something reasonable :P.

i also saw a very cool "survey of vision" talk by bill freeman... one of the big problems they talked about was that no one has a good p(image) prior model. the example given was that you usually have de-noising models like p(image)*p(noisy image|image) and you can weight p(image) by ^alpha... as alpha goes to zero, you should just get a copy of your noisy image... as alpha goes to infinity, you should end up getting a good image, maybe not the one you *want*, but an image nonetheless. this doesn't happen.

one way you can see that this doesn't happen is in the following task. take two images and overlay them. now try to separate the two. you *clearly* need a good prior p(image) to do this, since you've lost half your information.

i was thinking about what this would look like in language land. one option would be to take two sentences and randomly interleave their words, and try to separate them out. i actually think that we could solve this tasks pretty well. you could probably formulate it as a FST problem, backed by a big n-gram language model. alternatively, you could take two DOCUMENTS and randomly interleave their sentences, and try to separate them out. i think we would fail MISERABLY on this task, since it requires actually knowing what discourse structure looks like. a sentence n-gram model wouldn't work, i don't think. (although maybe it would? who knows.) anyway, i thought it was an interesting thought experiment. i'm trying to think if this is actually a real world problem... it reminds me a bit of a paper a year or so ago where they try to do something similar on IRC logs, where you try to track who is speaking when... you could also do something similar on movie transcripts.

hierarchical topic models with latent hierarchies drawn from the coalescent, kind of like hdp, but not quite. (yeah yeah i know i'm like a parrot with the coalescent, but it's pretty freaking awesome :P.)

That's it! Hope you all had a great holiday season, and enjoy your New Years (I know I'm going skiing. A lot. So there, Fernando! :)).

Anonymous said...

I've always looked at the image problem as an argument for posterior predictive checks rather than straight draws from the prior. It's possible that your original prior may be pretty diffuse but still puts probability enough mass on real images. Given data, the posterior should then be able to generate new images similar to the data, which is the standard textbook argument for posterior predictive checks (e.g. Gelman et al, 2003). Clearly all the current models fail. But posterior predictive checks should give hints about how to improve image models.

The same applies to language models. The prior doesn't need to generate sensible documents, but posterior predictive simulations should, given enough training data. Otherwise your model isn't rich enough and your prior doesn't put enough weight on true images.

Coming from a stats background, I've actually been surprised at how little iteration there is between posterior predictive checks and model building in computer science literature. This is a huge theme by statisticians doing applied Bayesian work in other fields. The payoff seems particularly big in CS applications because the models are so bad/hard.

Bob Carpenter said...

@Anonymous Computer scientists only tend to care about the predictive accuracy of their models.

My main beef is that they only tend to consider first-best predictions (e.g. 0/1 loss for classification) and not care about the probability assigned (e.g. log loss). This makes it hard to trade off recall for precision (or sensitivity for specificity) in an application, and most applications require either high precision or high recall.

Computer scientists don't usually evaluate individual parameters for significance or give them causal interpretations. That's because they're not interested in assessing the effect of education on income, but are rather interested in a single prediction such as "should I give this person a credit card?".

For language models, lack of predictive checks isn't so surprising when you consider that no one has ever built a language model (and I'm not talking just n-gram models here) that generates anything like sensible documents from the posterior predictive distribution.

You do see just this kind of posterior predictive checking in section 3 of Shannon's 1948 Mathematical Theory of Communication (yes, that's 61+ years ago) paper that introduced n-gram language models! What you see right away is the Markovian nature of n-gram models not representing long-term topical or syntactic consistency (as in Stephen Merrit's song title "Doris Day the Earth Stood Still").

On the other hand, you can do posterior predictive checks on smaller units than full docs. For instance, you could scatterplot expectated versus empirical counts of the next word given the previous word(s). You also see this in comparing prior coefficient distributions to posteriors (e.g. Goodman's paper on the Laplace [double exponential] prior).

hal said...

Anonymous: By "prior" I meant it in the Bayes' rule sense, not in the Bayesian sense... i.e., it is something like p(true image) which then gets corrupted into p(observation | true image). the "prior" then is, actually, a posterior given data, and it's that that doesn't generate anything remotely like images.

Analogously in NLP, as Bob says, a language model doesn't generate anything like sentences (see previous post of small changes begetting negative examples).

I actually think people do do a fair amount of something roughly analogous to posterior predictive simulations, but in the one-best sense that Bob doesn't like. That is, people run their models, see what they do, and make adjustments as appropriate. This is probably one of the major ways in which progress is made.

But Bob is totally right: I don't care at all if feature 18329 has x% effect on predicting whether a word is a determiner or not!

Back when I was a student, I took a class from Roni Rosenfeld where we had to build a system to disambiguate between true English sentences and sentences generated by a trigram language model. It's actually quite hard, until you start looking at using parsers and things like that. Nowadays I'd replace that with a fivegram and I bet it would be even more difficult. Of course, people do it with no effort at all (the bad ones "hurt" to read).

Fernando Pereira said...

Well, I've been skiing so hard the last two days, exploiting the bounty of yesterday's Tahoe storm, that I'm too tired to produce much in the way of technical comment. I'll just note that "I don't care at all if feature 18329 has x% effect on predicting whether a word is a determiner or not!" sounds a bit like sour grapes ;) If you had that information, it could help you debug your model when it goes badly wrong because of a change in the data distribution. Most academic ML work is not forced to deal with that critical issue because it is based on fixed datasets.

Anonymous said...

@Bob

That Shannon link is very nice. In statistics, as far as I can tell, Box (1980) and Rubin (1984) are viewed by many as the first clear statements of posterior predictive checks from a calibrated Bayes perspective, but the Shannon example is great; I'll definitely cite it from now on.

rr8004 said...

Very nice information. Thanks for this. Please come visit my site Colorado CO Phone Directory when you got time.

rr8004 said...

Very nice information. Thanks for this. Please come visit my site Aurora Phone Book when you got time.

Gustavo Lacerda said...

spelling quibble: Diaconis's first name is "Persi".

hal said...

fixed: thanks gustavo.

Anonymous said...
Montana Attorneys Legal Services said...

Hello, i am glad to read the whole content of this blog and am very excited and happy to say that the webmaster has done a very good job here to put all the information content and information at one place, i will must refer this information with reference on my website ...

Lawyers
Legal Services
, Attorneys
Directory
Directory

Unknown said...
Unknown said...
Unknown said...

one day i went shopping outside,and in an ed hardy store,I found some kinds of ed hardy i love most they are Your website is really good Thank you for the information ed hardy ed hardy ed hardy clothing ed hardy clothing ed hardy shoes ed hardy shoes don ed hardy don ed hardy ed hardy clothes ed hardy clothes ed hardy bags ed hardy bags ed hardy swimwear ed hardy swimwear ed hardy jeans ed hardy jeans ed hardy mens ed hardy mens Thank you for the information

Unknown said...
Unknown said...

Fashion trends change on daily basis, like Gold GHD. Following the latest in designer shades has become a passion of everyone, now Burberry Sunglasses. If you are the type of a woman who loves to explore in fashion, our ED Hardy Sunglasses will definitely satisfy your taste. Cheap Ed Hardy Sunglasses is also OK. Ed hardy streak of clothing is expanded into its wholesale ED Hardy T-shirt chain so that a large number of fans and users can enjoy the cheap ED Hardy Clothing range easily with the help of numerous secured websites, actually, our ED Hardy Outlet. As we all know, in fact discount ED Hardy, is based on the creations of the world renowned tattoo artist Don Ed Hardy. Well, this question is bound to strike the minds of all individuals. Many people may say Prada shoes is a joke, but we can give you Prada Sunglasses, because we have Prada handbags. Almost everyone will agree that Prada Purses are some of the most beautiful designer handbags marketed today. Now we have one new product: Prada totes. The reason is simple: fashion prohibited by ugg boots, in other words, we can say it as Cheap ugg boots. Would you like to wear Discount ugg boots. We have two kinds of fashionable boots: classic ugg boots and ugg classic tall boots. Ankh Royalty--the Cultural Revolution. Straightens out the collar, the epaulette epaulet, the Ankh Royalty Clothing two-row buckle. Now welcome to our Ankh Royalty Outlet. And these are different products that bear the most famous names in the world of fashion, like Ankh Royalty T shirt by the way -Prada, Spyder, Moncler(Moncler jackets，or you can say Moncler coats, Moncler T-shirt, Moncler vest，and you can buy them from our discount Moncler outlet), GHD, ED Hardy, Ankh Royalty, Twisted Heart.

Unknown said...

MBT will not only change,MBT boots,the way you use your MTB Shoes,What are the benefits?christian dior,Free shipping and free return shippingdior shoes,on all diorIncluded with each pair ofdior handbags,an instructional DVDWhat is Dior?dior sunglassesanother good paoduct Dior totesDo not worryThe urge to buy these goodsnew balancerevolutionary fitness aid from Swiss Masai，new balance shoeswhich may help reducenew balance outlet,transforms flat hardPuma Shoes, with top quality and cheap price. puma outlet,innovative sole design includes thePuma Sneaker,Here you can buy wide rangewholesale cl high heel sandalsquality and cheapMoncler，Very Cool, Comfortable and lightmoncler jacketsoriginal packingmoncler coatsYou might say that thediscount moncler vestNo one ever thought bothmoncler outletAs with everythingmoncler t-shirtThe new store north faceWe are offering you north face outletSome color combinations northfaceBut withinnorth face jacketsbecausediscount ugg bootsnorth face coats,cheap ugg bootsugg classic tall bootsugg boots,cheap ugg bootsclassic ugg bootsdiscount ugg bootswhich you are buying is uniqueclassic ugg bootsugg classic tall boots,ugg bootsGHD IV Salon Styler，We are offering you coach outlet，our store has been decorated coach handbags，Nicecoach totes

Unknown said...

Everyone has unique requirements in terms of brands,personal specifications and choices.Whether you want to clothing online,new moncler,moncler jackets,moncler coats,moncler outlet,moncler vest,moncler polot-shirt,spyder outlet,spyder polot-shirt,spyder vest,spyder coats,cheap ed hardy wholesale,ed hardy wholesale,discount ed hardy wholesale,wholesale ed hardy,ed hardy outletyou can just take your pick, put it in the shopping cart and make your payment.FromSexy Lingerie Store,Intimate Apparel,Sexy Halloween Costumes,Sexy Underwear,cheap vibram 5 fingers, discount vibram five fingers,Vibram running shoes,vibram five fingers shoes,vibram five finger outlet ,you get them all here. The real benefit is in the discount prices.

Unknown said...

Online shoe shopping has never been so easy.A one-stop destination for all your footwear needs!Nike Shox R4,Shox shoes,shox nz,ugg boots or new spyder,you name it and we have it.We are not only the premier shopping destination for spyder jacketsonline but also for people of all age-groups and for all brands. Buy online without leaving the comfort of your home. Whatever your needs: from clothing and accessories to all brands (cheap ugg boots,discount ugg boots,ugg boots,cheap ugg boots,discount ugg boots,Rare ghd,MBT boots,MBT shoes in fashion,cheap mbt shoes sale,discount mbt outlet 2010,MBT Walking Shoes)of shoes and sports gear,we has it all for you.

Unknown said...
Unknown said...
Anonymous said...

What a great post, I actually found it very thought provoking, you just never know sometimes when a golden nugget of information is going to land at your feet, thanks
New Hampshire title insurance, New Jersey title insurance, New Mexico title insurance, New York title insurance, North Carolina title insurance, North Dakota title insurance, Ohio title insurance, Oklahoma title insurance, Oregon title insurance, Pennsylvania title insurance, Rhode Island title insurance

Anonymous said...

A fantastic read….very literate and informative. Many thanks….what theme is this you are using and also, where is your RSS button ?
A fantastic read….very literate and informative. Many thanks….what theme is this you are using and also, where is your RSS button ?

Ann said...

Fashion trends change on daily basis, like Gold GHD. Now Burberry Sunglasses.Our ED Hardy Sunglasses will definitely satisfy your taste. ED Hardy T-shirt chain makes people enjoy the cheap ED Hardy Clothing range easily our ED Hardy Outlet. As we all know, in fact discount ED Hardy, is based. The reason is simple: fashion prohibited by polo boots, in other words, we can say it as polo shoes. Would you like to wear cheap ugg boots? Now welcome to our paul smith outlet. And these are different products, like Puma Shoes (or you can say Puma Sneaker, and you can buy them from our puma outlet), GHD, ED Hardy, UGG, Paul Smith and brand Sunglasses. For those who desire for a ugg boots but have to refrain themselves from buying one ugg boots, there are some company that offer discount ugg boots.

Unknown said...

Really trustworthy blog. Please keep updating with great posts like this one. I have booked marked your site and am about to email it to a few friends of mine that I know would enjoy reading..
sesli sohbetsesli chatkamerali sohbetseslisohbetsesli sohbet sitelerisesli chat siteleriseslichatsesli sohpetseslisohbet.comsesli chatsesli sohbetkamerali sohbetsesli chatsesli sohbetkamerali sohbet
seslisohbetsesli sohbetkamerali sohbetsesli chatsesli sohbetkamerali sohbet

Unknown said...

Really trustworthy blog. Please keep updating with great posts like this one. I have booked marked your site and am about to email it to a few friends of mine that I know would enjoy reading..
sesli sohbetsesli chat
sesli sohbet siteleri

sesli chat siteleri sesli sohbetsesli chat
sesli sohbet siteleri
sesli chat siteleri
SesliChat
cÄ±lgÄ±n sohbet
gÃ¼zel kÄ±zlar
bekar kÄ±zlar
dul bayanlar
seviyeli insanlar
yarÄ±ÅŸma
canlÄ± mÃ¼zik
izdivac
en gÃ¼zel evlilik
seslisohbet odalari
Sesli Chat
SesliChat Siteleri
Sesli Chat sitesi
SesliChat sitesi
SesliSohbet
Sesli Sohbet
Sesli Sohbet Sitesi
SesliSohbet Sitesi
SesliSohbet Siteleri
Muhabbet Sitesi
kamerali chat
GÃ¶rÃ¼ntÃ¼lÃ¼ Sohbet
Hasret gÃ¼lleri
Ã‡et sitesi
SesliSohbet
Sesli Sohbet
Canli sohbet
Turkce sohbet
Kurtce Sohbet
Kurtce Chat
Kurtce Muhabbet
Kurtce Sohbet
Kurdish Chat
SesliChat
Sesli Chat
SesliSanal
Guncel Haber
sohbet Sitesi
Chat sitesi..

Unknown said...

kAriZmA > HitSesLi.Com
Ortak KÃ¼ltÃ¼rlerin BuluÅŸma Noktasi >
GÃ¶rÃ¼nÃ¼tÃ¼lÃ¼ Sohbet YariÅŸma EÄŸlence CanlÄ± MÃ¼zik KÄ±zlar Para Kazan Hersey Burada Sen Nerdesin
Really trustworthy blog. Please keep updating with great posts like this one. I have booked marked your site and am about to email it to a few friends of mine that I know would enjoy reading..
sesli sohbetsesli chat
sesli sohbet siteleri

sesli chat siteleri sesli sohbetsesli chat
sesli sohbet siteleri
sesli chat siteleri
SesliChat
cÄ±lgÄ±n sohbet
gÃ¼zel kÄ±zlar
bekar kÄ±zlar
dul bayanlar
seviyeli insanlar
yarÄ±ÅŸma
canlÄ± mÃ¼zik
izdivac
en gÃ¼zel evlilik
sesliparti
seslisohbet odalari
Sesli Chat
SesliChat Siteleri
Sesli Chat sitesi
SesliChat sitesi
SesliSohbet
Sesli Sohbet
Sesli Sohbet Sitesi
SesliSohbet Sitesi
SesliSohbet Siteleri
Muhabbet Sitesi
kamerali chat
GÃ¶rÃ¼ntÃ¼lÃ¼ Sohbet
Hasret gÃ¼lleri
Ã‡et sitesi
SesliSohbet
Sesli Sohbet
Canli sohbet
Turkce sohbet
Kurtce Sohbet
Kurtce Chat
Kurtce Muhabbet
Kurtce Sohbet
Kurdish Chat
SesliChat
Sesli Chat
SesliSanal
Guncel Haber
sohbet Sitesi
Chat sitesi..

Unknown said...

NewStreetFashion
Ed Hardy
stylish design
Ed Hardy Wholesale
fashion excellent quality
wholesale Ed Hardy
ED Hardy clothing bring you a super surprise!
ed hardy wholesale clothing
The quality is so good
christian audigier
if you really want it
jordan 8
jordan 9
jordan 10

coatjack said...

coats & jackets
If one knows where to, the world will give way.
north face jackets
moncler
World-class brands
moncler jackets
The top fabric produced in France is 100%
duvetica
Italy's top outdoor brands
peuterey
Good quality and function, more accord with fashionable design
spyder jackets
High-grade, innovation, the trend of the brand
Columbia Sportswear
The quality is so good
quiksilver jackets
Young and creative style
ski jacket

You can have a look at it.
jordan shoes
Wow.
jordan 7
Wonderful!
air yeezy
jordan true flight
jordan 3
jordan 4
We offer different styles.
jordan 1
jordan 2
Thanks.
jordan 5
jordan ajf shoes
There are cheap shoes to choose
nike footwear
jordan flight 45
Good quality with low price.
air jordan 2010
Air Jordan 2009
Enjoy it!
jumpman
nike trainers

Anonymous said...

I can't believe it.
abercrombie outlet
You can have a look at it.
abercrombie fitch outlet
The quality is so good
nike shoes wholesale
wholesale jordan shoes
Young and creative style
wholesale clothing
wholesale nike shoes
wholesale jordans
Young and creative style
china wholesale
There are cheap shoes to choose
jordan 6
jordan 28

Unknown said...

Really trustworthy blog. Please keep updating with great posts like this one. I have booked marked your site and am about to email it to a few friends of mine that I know would enjoy reading..

sesli sohbet
seslisohbet
sesli chat
seslichat
sesli sohbet sitesi
sesli chat sitesi
sesli sohpet
kamerali sohbet
kamerali chat
webcam sohbet

jacket said...

You can have a look at it.
coats & jackets
jordan shoes
The quality is so good.
abercrombie and fitch
abercrombie & fitch
Abercrombie and fitch outle

combattery84 said...
combattery84 said...
combattery84 said...
ylinling001 said...

I like your article, really interesting! My point is also very good, I hope you'll like:chi flat iron are a very popular choice of hair straightener.New Balance,new Blance shoes,new Blance Outlet are some of the most comfortable and stylish shoes on the market today. The designer has a whole range of shoes for all types of athletes. five finger shoes,vibram five fingers,Five fingers shoes give women the feeling of walking barefoot while still keeping the feet protected.

Anonymous said...

Really trustworthy blog. Please keep updating with great posts like this one. I have booked marked your site and am about to email it

to a few friends of mine that I know would enjoy reading..
seslisohbet
seslichat
sesli sohbet
sesli chat
sesli
sesli site
gÃ¶rÃ¼nlÃ¼tÃ¼ sohbet
gÃ¶rÃ¼ntÃ¼lÃ¼ chat
kameralÄ± sohbet
kameralÄ± chat
sesli sohbet siteleri
sesli chat siteleri
gÃ¶rÃ¼ntÃ¼lÃ¼ sohbet siteleri
gÃ¶rÃ¼ntÃ¼lÃ¼ chat siteleri
kameralÄ± sohbet siteleri
canlÄ± sohbet
sesli muhabbet
gÃ¶rÃ¼ntÃ¼lÃ¼ muhabbet
kameralÄ± muhabbet
seslidunya
seslisehir
sesli sex

trustme said...

This is really a nice blog, I appreciate you for telling us so nice things, thank you!By the way, if you like nike tn you can come here to choose! We have a lot of
nike tn,tn chaussures,
nike tn chaussures
nike tn requin chaussures,nike air max tn chaussures.
nike homme chaussures,
nike femme chaususres,
nike enfant chaussres,
MBT France
vibram
If you want to find the shoes according to the sorts, then here you can have the informations,
we classied the shoes in nike presto,
nike air max,
nike air rift ninja,
tn requin,tn pas cher
vibram fivefingers,
converse.
At the same time, the vibram also offer you in our store.
You also can choose the most fashionable sunglasses here, it really can make you different from other people. We have
sunglasses,designer sunglasses,
wholesale sunglasses,sunglasses discount in USA.
They includ men's sunglasses,women's sunglasses.
So many fashion brands are for you,like Dior Sunglasses,
Emporio Armani Sunglasses,
Fendi Sunglasses,
Giorgio Armani Sunglasses,
Gucci Sunglasses,
LV Sunglasses and so on.

Unknown said...

Really trustworthy blog. Please keep updating with great posts like this one. I have booked marked your site and am about to email it

to a few friends of mine that I know would enjoy reading..
seslisohbet
seslichat
sesli sohbet
sesli chat
sesli
sesli site
gÃ¶rÃ¼nlÃ¼tÃ¼ sohbet
gÃ¶rÃ¼ntÃ¼lÃ¼ chat
kameralÄ± sohbet
kameralÄ± chat
sesli sohbet siteleri
sesli chat siteleri
sesli muhabbet siteleri
gÃ¶rÃ¼ntÃ¼lÃ¼ sohbet siteleri
gÃ¶rÃ¼ntÃ¼lÃ¼ chat siteleri
gÃ¶rÃ¼ntÃ¼lÃ¼ muhabbet siteleri
kameralÄ± sohbet siteleri
kameralÄ± chat siteleri
kameralÄ± muhabbet siteleri
canlÄ± sohbet
sesli muhabbet
gÃ¶rÃ¼ntÃ¼lÃ¼ muhabbet
kameralÄ± muhabbet
birsesver
birses
seslidunya
seslisehir
sesli sex