20 June 2009

Why I Don't Buy Clustering Axioms

In NIPS 15, Jon Kleinberg presented some impossibility results for clustering. The idea is to specify three axioms that all clustering functions should obey and examine those axioms.

Let (X,d) be a metric space (so X is a discrete set of points and d is a metric over the points of X). A clustering function F takes d as input and produces a clustering of the data. The three axioms Jon states that all clustering functions should satisfy are:

  1. Scale invariance: For all d, for all a>0, F(ad) = F(d). In other words, if I transform all my distances by scaling them uniformly, then the output of my clustering function should stay the same.
  2. Richness: The range of F is the set of all partitions. In other words, there isn't any bias that prohibits us from producing some particular partition.
  3. Consistency: Suppose F(d) gives some clustering, C. Now, modify d by shrinking distances within clusters of C and expanding distances between clusters in C. Call the new metric d'. Then F(d') = C.
Kleinberg's result is that there is no function F that simultaneously satisfies all these requirements. Functions can satisfy two, but never all three. There have been a bunch of follow on papers, including one at NIPS last year and one that I just saw at UAI.

If you think about these axioms a little bit, they kind of make sense. My problem is that if you think about them a lot of bit, you (or at least I) realize that they're broken. The biggest broken one is consistency, which becomes even more broken when combined with scale invariance.

What I'm going to do to convince you that consistency is broken is start with some data in which there is (what I consider) a natural clustering into two clusters. I'll then apply consistency a few times to get something that (I think) should yield a different clustering.

Let's start with some data. The colors are my opinion as to how the data should be clustered:

I hope you agree with my color coding. Now, let's apply consistency. In particular, let's move some of the red points, only reducing inter-clustering distances. Formally, we find the closest pair of points and move things toward those.

The arrows denote the directions these points will be moved. To make the situation more visually appealing, let's move things into lines:
Okay, this is already looking funky. Let's make it even worse. Let's apply consistency again and start moving some blue points:
Again, let's organize these into a line:
And if I had given you this data to start with, my guess is the clustering you'd have come up with is more like:
This is a violation of consistency.

So, what I'd like someone to do is to argue to my why consistency is actually a desirable property.

I can come up with lots of other examples. One reason why this invariance is bad is because it renders the notion of "reference sizes" irrelevant. This is of course a problem if you have prior knowledge (eg., one thing measured in millimeters, the other in kilometers). But even in the case where you don't know knowledge, what you can do is take the following. Take data generated by thousands of well separated Gaussians, so that the clearly right thing to do is have one cluster per Gaussian. Now, for each of these clusters except for one, shrink them down to single points. This is possible by consistency. Now, your data basically looks like thousands-1 of clusters with zero inter-cluster distances and then one cluster that's spread out. But now it seems that the reasonable thing is to put each data point that was in this old cluster into its own cluster, essentially because I feel like the other data shows you what clusters should look like. If you're not happy with this, apply scaling and push these points out super far from each other. (I don't think this example is as compelling as the one I drew in pictures, but I still think it's reasonable.

Now, in the UAI paper this year, they show that if you fix the number of clusters, these axioms are now consistent. (Perhaps this has to do with the fact that all of my "weird" examples change the number of clusters -- though frankly I don't think this is necessary... I probably could have arranged it so that the resulting green and blue clusters look like a single line that maybe should just be one cluster by itself.) But I still feel like consistency isn't even something we want.

(Thanks to the algorithms group at Utah for discussions related to some of these issues.)

UPDATE 20 JUNE 2009, 3:49PM EST

Here's some data to justify the "bad things happen even when the number of clusters stays the same" claim.

Start with this data:


Now, move some points toward the middle (note they have to spread to the side a bit so as not to decrease intra-cluster distances).

Yielding data like the following:

Now, I feel like two horizontal clusters are most natural here. But you may disagree. What if I add some more data (ie., this is data that would have been in the original data set too, where it clearly would have been a third cluster):

And if you still disagree, well then I guess that's fine. But what if there were hundreds of other clusters like that.

I guess the thing that bugs me is that I seem to like clusters that have similar structures. Even if some of these bars were rotated arbitrarily (or, preferably, in an axis-aligned manner), I would still feel like there's some information getting shared across the clusters.

61 comments:

Anonymous said...

The richness property also seems strange. In supervised learning we know that the learner has to have an inductive bias, otherwise it cannot generalize beyond what it has already seen. Shouldn't the case be the same in unsupervised learning? Shouldn't the clustering algorithm prefer some partitionings over the others?

hal said...

You're interpreting richness too strongly. All it says is that anything is possible. Our algorithm may be biased in any way it sees fit. (For example, most clustering algorithms are probably biased toward fewer clusters. This is probably good.)

hal said...

Okay, I figured I should probably justify the "this is bad even when the number of clusters stays the same" thing. I've modified the post.

joe said...

is part of the issue distinguishing between what a pattern (structure) and a cluster is? is there a deep distinction between the two?

it seems that the issue you have with the clustering axioms is that they allow transformations that lead to an alignment of data points in some pattern (like two lines), but in general, how should structure be defined and found over collections of data points? i think this has interesting connections to the perception of structure. matt dry, dan navarro and others have a paper in this year's cog sci comparing human clusterings to some simple machine learning algorithms (paper). it seems it would be interesting to compare how people cluster collections of points to machine learning algorithms. (there's already a wide body of work on how people cluster collections of point from a purely psychological point of view as well).

Reza Bosagh Zadeh said...

It's important to note three things:

1) Some of the examples shown are not proper Consistent changes. The very first change you make in the counter-examples is actually not a valid Consistent change. This is because even though you are making points in the same clusters closer together, you are also making them closer to points in the other cluster. Specifically, by moving some of the bottom red points upwards, you are making them closer to some of the upper blue points. This is an illegal move with respect to Consistency.

2) Let us assume the changes were indeed Consistent ones, and you get the lines in the very last example. Once the Consistent change is finished, you seem to be asking yourself the question: "What is the most natural clustering here?". This is actually not the right question to be asking. Instead, you should be asking: "What is the most natural clustering here, GIVEN that the previous answer of the algorithm?". Consistency is not an axiom that helps you decide what is natural is natural, but rather it is there to ensure the clustering algorithm is at least self-affirming its own previous answers.

3) There is some confusion produced by thinking about this in 2d, or indeed any metric embedding. A more natural way to convince oneself is to think about an abstract distance matrix, and what consistent changes mean in that setting. When the number of clusters is fixed, Consistency is fairly non-contraversial. Encouraging evidence for this is that popular algorithms such as Single-Linkage (hierarchical) and the Min-Sum objective satisfy Consistency.

Reza Bosagh Zadeh said...

Slight typo in the last comment:

"Consistency is not an axiom that helps you decide what is a natural clustering, but rather it is there to ensure the clustering algorithm is at least self-affirming its own previous answers."

Mariana Soffer said...

One idea is to "guess" or learn the distribution each cluster has, and with that knowledge perform the clustering process respecting in your calculations the cluster distributions, which might alter
your distance functions depending where you are located in space. Of course the distribution can be different in each dimension the data has.

Sandy Peterson said...

Hello mate, I want to thank you for this nice blog. Would you mind telling me some secrets for a succesful blog ? Which could attract some visitors than it normally does. Please come visit my site Adoption Agencies Directory when you got time.

Neil Toronto said...

Reza: "Consistency is not an axiom that helps you decide what is a natural clustering, but rather it is there to ensure the clustering algorithm is at least self-affirming its own previous answers."

Hal's "first new look" intuition seems to me to be consistent with the math. It perhaps looks otherwise to you because he didn't specify that he was using the same "natural to him" clustering algorithm.

Hal: Are you sure it's allowable to argue against axiom 3 by positing new data sets?

Making axiom 3 more explicit should help. It's not this:

3. F(d) = C = F(d')

It's really this:

3'. G(d, X) = C = G(d', X)

with the appropriate conditions on d', and X is a data set. G is like F, but the data set it operates on is explicit. (I suppose F could have been a closure; in that case I've defunctionalized it.) It appears that you're arguing against this axiom:

3-hal. G(d, X) = C = G(d, X')

It's not clear to me that either axiom 3-hal or 3' is sufficient (along with 1 and 2) to prove the theorem, or that 3-hal and 3' are equivalent.

A second question, related to angle of attack. How general is this formulation of clustering? Can "G : Metric x Dataset -> Clustering" describe any clustering algorithm? Does it describe most popular clustering algorithms? Are there any known that can't be formulated like this?

Night Web Directory said...

Hello mate, I want to thank you for this nice blog. Would you mind telling me some secrets for a succesful blog ? Which could attract some visitors than it normally does. Please come visit my site FreeAdultEntertainment when you got time.

Sioux Falls South Dakota Business Directory said...

This is just another reason why I like your website. I like your style of writing you tell your stories without out sending us to 5 other sites to complete the story. Please come visit my site Sioux Falls South Dakota Business Directory when you got time.

Pimpin Kicks said...

Wholesale shoes direct from the factory. Wholesale shoes,, Nike shoes, nike af1, nike shox, nike r4, nike rt1, ATO, BAPE, Bathing Ape, Puma Selling Dunk

jyoti said...

This article was extremely interesting, especially since I was searching for thoughts on this subject last Thursday. Please come visit my site Chula Vista Business Directory when you got time.

jyoti said...

It is very interesting article and quite impressive and more informative and looking forward to read such article. Please come visit my site Modesto Yellow Page Business Directory when you got time

Roofing Contractors said...

Me and my friend were arguing about an issue similar to this! Now I know that I was right. lol! Thanks for the information you post. Please come visit my site Roofing when you got time.

Neon Sign said...

What a blog filled with vital and important information this is .. It must have taken a lot of hours for you to write these yourself. Hats off from me for your hard work. Please come visit my site
Signage when you got time.

poll said...

I was thinking of looking up some of them newspaper websites, but am glad I came here instead. Although glad is not quite the right word… let me just say I needed this after the incessant chatter in the media, and am grateful to you for articulating something many of us are feeling - even from distant shores. Please come visit my site Phone Directory Of Newark City New Jersey NJ State when you got time.

poll said...

You may have not intended to do so, but I think you have managed to express the state of mind that a lot of people are in. The sense of wanting to help, but not knowing how or where, is something a lot of us are going through. Please come visit my site Local Business Directory Of Jersey City U.S.A. when you got time.

ass said...

In response to multi-era, Burberry will also be designed to reach extends to other areas, such as men's burberry polo shirt, ladies burbrery polos and babolat aeropro drive. A period of change is not just for the Burberry company is celebrating challenges, and it also gives the same with a change in the company. Puma basket and ed hardy clothes is also a product of a new era, the younger generation on the Cheap puma shoes worship has reached frenzied proportions. Wholesale Handbagsand cheap tennis racquets have become a way of reducing consumption cost.

ass said...

James Dean wore in the blockbuster film Levi jeans and Lacoste polo shirts,Levis Jeans from a "personality and rebellion" symbol. While their counterparts in another company to do jeans Diesel, it has more than the Diesel Jeans another culture, fashion spokesperson. In a series of autumn and winter are filled with a thick knighthood, while below it a cheap jacke is the best proof of this. puma basket t with a nike max shoes make you full of self-confidence. ed hardy shirts make you feel the freedom of street culture with nature.

ass said...

www.crazypurchase.com Web site for the majority of the china wholesalers, and buy products wholesale to provide information and buy wholesale from china information exchange platform dedicated to becoming China Wholesale.

poll said...

that your great job is clearly identifed. I was wondering if you offer any subscription to your RSS feeds as I would be very interested and can’t find any link to subscribe here. Please come visit my site Phone Directory Of Chula Vista City California CA State when you got time.

poll said...

This is just another reason why I like your website. I like your style of writing you tell your stories without out sending us to 5 other sites to complete the story. Please come visit my site Modesto Yellow Pages when you got time.

rr8004 said...

I really liked your blog! Please come visit my site Nashville Business Directory when you got time.

rr8004 said...

I really liked your blog! Please come visit my site Nashville Yellow Page Business Directory when you got time.

rr8004 said...

I enjoyed reading your work! GREAT post! I looked around for this… but I found you! Anyway, would you mind if I threw up a backlink from my site? Please come visit my site San Francisco Business Services And Classifieds when you got time.

rr8004 said...

I enjoyed reading your work! GREAT post! I looked around for this… but I found you! Anyway, would you mind if I threw up a backlink from my site? Please come visit my site California CA Phone Directory when you got time.

Kevembuangga said...

So you expected tinkering to have rules and you are disappointed it has none?
Or do you believe in "meaning for free" à la Chris Anderson?
I see no reason why a randomly picked and massaged metric is supposed to highlight significant features.

rr8004 said...

Thank you for your good humor and for allowing yourself to be convinced that this was the right show for you to work on. Please come visit my site Bakersfield Yellow Page Business Directory when you got time.

rr8004 said...

Thank you for your good humor and for allowing yourself to be convinced that this was the right show for you to work on. Please come visit my site Local Business Directory Of Bakersfield U.S.A. when you got time.

Anonymous said...

o‰ï‚¢‹i’ƒo‰ï‚¢ƒJƒtƒFƒeƒŒƒNƒ‰•s—σZƒbƒNƒXƒtƒŒƒ“ƒhƒZƒtƒŒo‰ï‚¢o‰ï‚¢o‰ï‚¢ŒfŽ¦”o‰ï‚¢o‰ï‚¢o‰ï‚¢lÈ•—‘­ƒfƒŠƒwƒ‹ƒfƒŠƒoƒŠ[ƒwƒ‹ƒXo‰ï‚¢o‰ï‚¢–³—¿ƒtƒBƒŠƒsƒ“ƒ‰ƒCƒuƒ`ƒƒƒbƒgƒAƒ_ƒ‹ƒgƒ‰ƒCƒuƒ`ƒƒƒbƒgƒfƒŠƒwƒ‹

Anonymous said...

•s“®ŽYƒ\[ƒvƒ‰ƒ“ƒhƒAƒNƒZƒXƒJƒEƒ“ƒ^[ƒRƒŒƒXƒeƒ[ƒ‹’†«Ž‰–b‰Ô•²ÇÝ‘[ƒN“àEÝ‘îƒAƒ‹ƒoƒCƒgŠ£‘‡”§ƒ_ƒCƒGƒbƒg@HŽ–ƒTƒvƒŠƒƒ“ƒg–³—¿è‚¢o‰ï‚¢ŽRŒûƒNƒŒƒWƒbƒgƒJ[ƒhŒ»‹à‰»ƒNƒŒƒWƒbƒgŒ»‹à‰»ƒ‰ƒCƒuƒ`ƒƒƒbƒgƒtƒBƒŠƒsƒ“ƒ`ƒƒƒbƒgƒŒƒfƒBƒpƒ\ƒRƒ“Ý‘[ƒN

Anonymous said...

ƒjƒLƒrCholesterol…’Ž—₦«‚Þ‚­‚ÝŽYŒã‚í‚«‚ªŽÔ”ƒŽæ“]E‰Á—îL“V‘RƒIƒŠƒS“œ’ܐ…’Ž‹ž“s‚«‚à‚̍ݑîŽdŽ–

Anonymous said...

o‰ï‚¢ŒF–{o‰ï‚¢‹ž“so‰ï‚¢ŽOdo‰ï‚¢‹{éo‰ï‚¢‹{èo‰ï‚¢’·–ìo‰ï‚¢’·èo‰ï‚¢“ޗǏo‰ï‚¢VŠƒo‰ï‚¢‘啪o‰ï‚¢‰ªŽRo‰ï‚¢‰«“êo‰ï‚¢‘åão‰ï‚¢²‰êo‰ï‚¢é‹Êo‰ï‚¢Ž ‰êo‰ï‚¢“‡ªo‰ï‚¢Ã‰ªo‰ï‚¢“Ȗ؏o‰ï‚¢“¿“‡

Anonymous said...

o‰ï‚¢“Œ‹žo‰ï‚¢’¹Žæo‰ï‚¢•xŽRo‰ï‚¢˜a‰ÌŽRo‰ï‚¢ŽRŒ`o‰ï‚¢ŽRŒûo‰ï‚¢ŽR—œo‰ï‚¢–k‹ãBo‰ï‚¢‰ºŠÖo‰ï‚¢ìèo‰ï‚¢_ŒËo‰ï‚¢‹v—¯•Äo‰ï‚¢…ŒËo‰ï‚¢–¼ŒÃ‰®o‰ï‚¢‘å–´“co‰ï‚¢é‹Êo‰ï‚¢äo‰ï‚¢å‘äo‰ï‚¢‰¡•lo‰ï‚¢‰¡{‰êo‰ï‚¢ŽD–yo‰ï‚¢ìè

Anonymous said...

•Ÿˆäo‰ï‚¢ˆ¤’mo‰ï‚¢Šò•Œo‰ï‚¢Ã‰ªo‰ï‚¢ŽOdo‰ï‚¢•ºŒÉo‰ï‚¢‘åão‰ï‚¢˜a‰ÌŽRo‰ï‚¢Ž ‰êo‰ï‚¢‹ž“so‰ï‚¢“ޗǏo‰ï‚¢ŽRŒûo‰ï‚¢’¹Žæo‰ï‚¢“‡ªo‰ï‚¢‰ªŽRo‰ï‚¢L“‡o‰ï‚¢“¿“‡o‰ï‚¢ìo‰ï‚¢ˆ¤•Qo‰ï‚¢‚’mo‰ï‚¢

Anonymous said...

o‰ï‚¢ŒF–{o‰ï‚¢‹ž“so‰ï‚¢ŽOdo‰ï‚¢‹{éo‰ï‚¢‹{èo‰ï‚¢’·–ìo‰ï‚¢’·èo‰ï‚¢“ޗǏo‰ï‚¢VŠƒo‰ï‚¢‘啪o‰ï‚¢‰ªŽRo‰ï‚¢‰«“êo‰ï‚¢‘åão‰ï‚¢²‰êo‰ï‚¢é‹Êo‰ï‚¢Ž ‰êo‰ï‚¢“‡ªo‰ï‚¢Ã‰ªo‰ï‚¢“Ȗ؏o‰ï‚¢“¿“‡

Anonymous said...

•Ÿˆäo‰ï‚¢ƒJƒtƒFˆ¤’mo‰ï‚¢ƒJƒtƒFŠò•Œo‰ï‚¢ƒJƒtƒFÃ‰ªo‰ï‚¢ƒJƒtƒFŽOdo‰ï‚¢ƒJƒtƒF•ºŒÉo‰ï‚¢ƒJƒtƒF‘åão‰ï‚¢ƒJƒtƒF˜a‰ÌŽRo‰ï‚¢ƒJƒtƒFŽ ‰êo‰ï‚¢ƒJƒtƒF‹ž“so‰ï‚¢ƒJƒtƒF“ޗǏo‰ï‚¢ƒJƒtƒFŽRŒûo‰ï‚¢ƒJƒtƒF’¹Žæo‰ï‚¢ƒJƒtƒF“‡ªo‰ï‚¢ƒJƒtƒF‰ªŽRo‰ï‚¢ƒJƒtƒFL“‡o‰ï‚¢ƒJƒtƒF“¿“‡o‰ï‚¢ƒJƒtƒFìo‰ï‚¢ƒJƒtƒFˆ¤•Qo‰ï‚¢ƒJƒtƒF‚’mo‰ï‚¢ƒJƒtƒF

Anonymous said...

•ŸˆäƒeƒŒƒNƒ‰ˆ¤’mƒeƒŒƒNƒ‰Šò•ŒƒeƒŒƒNƒ‰Ã‰ªƒeƒŒƒNƒ‰ŽOdƒeƒŒƒNƒ‰•ºŒÉƒeƒŒƒNƒ‰‘åãƒeƒŒƒNƒ‰˜a‰ÌŽRƒeƒŒƒNƒ‰Ž ‰êƒeƒŒƒNƒ‰‹ž“sƒeƒŒƒNƒ‰“ޗǃeƒŒƒNƒ‰ŽRŒûƒeƒŒƒNƒ‰’¹ŽæƒeƒŒƒNƒ‰“‡ªƒeƒŒƒNƒ‰‰ªŽRƒeƒŒƒNƒ‰L“‡ƒeƒŒƒNƒ‰“¿“‡ƒeƒŒƒNƒ‰ìƒeƒŒƒNƒ‰ˆ¤•QƒeƒŒƒNƒ‰‚’mƒeƒŒƒNƒ‰

Anonymous said...

•ŸˆäƒZƒtƒŒˆ¤’mƒZƒtƒŒŠò•ŒƒZƒtƒŒÃ‰ªƒZƒtƒŒŽOdƒZƒtƒŒ•ºŒÉƒZƒtƒŒ‘åãƒZƒtƒŒ˜a‰ÌŽRƒZƒtƒŒŽ ‰êƒZƒtƒŒ‹ž“sƒZƒtƒŒ“ޗǃZƒtƒŒŽRŒûƒZƒtƒŒ’¹ŽæƒZƒtƒŒ“‡ªƒZƒtƒŒ‰ªŽRƒZƒtƒŒL“‡ƒZƒtƒŒ“¿“‡ƒZƒtƒŒìƒZƒtƒŒˆ¤•QƒZƒtƒŒ‚’mƒZƒtƒŒ

Anonymous said...

•Ÿˆä•s—ψ¤’m•s—ÏŠò•Œ•s—ϐɪ•s—ÏŽOd•s—Ï•ºŒÉ•s—Ï‘åã•s—Ϙa‰ÌŽR•s—ÏŽ ‰ê•s—Ï‹ž“s•s—Ï“Þ—Ç•s—ÏŽRŒû•s—Ï’¹Žæ•s—Ï“‡ª•s—ωªŽR•s—ύL“‡•s—Ï“¿“‡•s—ύì•s—ψ¤•Q•s—ύ‚’m•s—Ï

Anonymous said...

—D—Ǐo‰ï‚¢ƒTƒCƒgƒAƒ_ƒ‹ƒg“ŽB‘fln—ƒGƒƒAƒjƒAV——Doˆ§‚¢o‡‚¢ŒnƒTƒCƒg‚Å‚ ‚¢deai•s—Ï•s—Ï‚µ‚Ä‚Ý‚Ü‚¹‚ñ‚©•s—σp[ƒgƒi[•s—ψ¤l2ƒVƒ‡ƒbƒgƒ`ƒƒƒbƒgo‰ï‚¢•s—ϐlÈƒNƒŒƒWƒbƒgƒJ[ƒh

rr8004 said...

I wanted to thank you for this great read!! I definitely enjoying every little bit of it :) I have you bookmarked to check out new stuff you post. Please visit my Please visit my blog when you have time Please come visit my site City Guide Reno when you got time.

rr8004 said...

I wanted to thank you for this great read!! I definitely enjoying every little bit of it :) I have you bookmarked to check out new stuff you post. Please visit my Please visit my blog when you have time Please come visit my site Business Resources Comprehensive Listings when you got time.

Godar said...

There was this guy who believed very much in true love and decided to take his time to wait for his right girl to appear.
nike shox nz shoes
cheap nike shox nz shoes
nike shoxs
Chaussures puma
wholesale nike shox nz shoes
cheap ugg boots
nike womens shoes
hommes nike chaussures
femmes nike
mens puma shoes
hommes puma chaussures
femmes puma chaussures
Nike Air Max 360 chaussures
Nike Air Max 90 Chaussures
Nike Air Max 95 Chaussures
Nike Air Max Ltd Chaussures
nike shox
Nike Max Tn Chaussures
nike 360 air max
nike running shoes
NIKE air shoes
nike shox nz shoes online store
nike air max
Nike Air TN Spider Chaussures
Nike Max Plata Chaussures
Nike R4 Plating Chaussures
Nike Shox Rival Velcro
Nike Shox Deliver Chaussures
Nike Shox Classic Chaussures
Nike NZ Plating Chaussures
wholesale nike shoes
nike shox torch
sneakers shoes
Nike Tn
discount nike shoes
nike shox r4
tn dollar
cheap nike shoes
nike tennis shoes
cheap nike shox
free shipping shoes
Paypal Credit card Accept
nike shoes
nike discount shoes
cheap puma shoes
nike shox shoes
chaussures nike
nike free shoes
buy shoes online
You may painfully regret, only to realise that it is too late.

Masters Research Paper said...

Many institutions limit access to their online information. Making this information available will be an asset to all.

tariely said...

Классные мультики мультфильмы бесплатно на кинозоуне.
электронная почта без регистрации

tar said...

электронная почта без регистрации

gamefan12 said...

I do not buy Clustering Axioms either. I think you are so right about this.
denver mesothelioma lawyers

qishaya said...

one day i went shopping outside,and in an ed hardy store,I found some kinds of ed hardy i love most they are Your website is really good Thank you for the information ed hardy ed hardy ed hardy clothing ed hardy clothing ed hardy shoes ed hardy shoes don ed hardy don ed hardy ed hardy clothes ed hardy clothes ed hardy bags ed hardy bags ed hardy swimwear ed hardy swimwear ed hardy jeans ed hardy jeans ed hardy mens ed hardy mens Thank you for the information

seldamuratim said...

Really trustworthy blog. Please keep updating with great posts like this one. I have booked marked your site and am about to email it to a few friends of mine that I know would enjoy reading..
sesli sohbetsesli chatkamerali sohbetseslisohbetsesli sohbet sitelerisesli chat siteleriseslichatsesli sohpetseslisohbet.comsesli chatsesli sohbetkamerali sohbetsesli chatsesli sohbetkamerali sohbet
seslisohbetsesli sohbetkamerali sohbetsesli chatsesli sohbetkamerali sohbet

seldamuratim said...

Really trustworthy blog. Please keep updating with great posts like this one. I have booked marked your site and am about to email it to a few friends of mine that I know would enjoy reading..

sesli sohbet
seslisohbet
sesli chat
seslichat
sesli sohbet sitesi
sesli chat sitesi
sesli sohpet
kamerali sohbet
kamerali chat
webcam sohbet

combattery84 said...

Laptop Battery
ACER Laptop Battery
ASUS Laptop Battery
COMPAQ Laptop Battery
Dell Laptop Battery
FUJITSU Laptop Battery
HP Laptop Battery
IBM Laptop Battery
SONY Laptop Battery
TOSHIBA Laptop Battery
APPLE M8403 battery
APPLE A1078 Battery
APPLE A1079 battery
APPLE A1175 battery 1
APPLE a1185 battery
APPLE A1189 battery
Acer aspire 5920 battery
Acer btp-arj1 battery
Acer LC.BTP01.013 battery
Acer ASPIRE 1300 battery
Acer ASPIRE 1310 battery
Acer Aspire 1410 battery
Acer ASPIRE 1680 battery
ACER BTP-63D1 battery
ACER BTP-43D1 battery
Acer lc.btp05.001 battery
Acer aspire 3000 battery
Acer Travelmate 4000 battery
ACER aspire 5560 battery
ACER BATBL50L6 battery
ACER TravelMate 240 Battery

combattery84 said...

Dell inspiron 8500 battery
Dell Inspiron 4100 battery
Dell Inspiron 4000 battery
Dell Inspiron 8200 battery
Dell FK890 battery
Dell Inspiron 1721 battery
Dell Inspiron 1300 Battery
Dell Inspiron 1520 Battery
Dell Latitude D600 Battery
Dell XPS M1330 battery
Dell Latitude D531N Battery
Dell INSPIRON 6000 battery
Dell INSPIRON 6400 Battery
Dell Inspiron 9300 battery
Dell INSPIRON 9400 Battery
Dell INSPIRON e1505 battery
Dell INSPIRON 2500 battery
Dell INSPIRON 630m battery
Dell Latitude D820 battery
Dell Latitude D610 Battery
Dell Latitude D620 battery
Dell Latitude D630 battery
Dell xps m1210 battery
Dell e1705 battery
Dell d830 battery
Dell inspiron 2200 battery
Dell inspiron 640m battery
Dell inspiron b120 battery
Dell xps m1210 battery

combattery84 said...

Dell inspiron xps m1710 battery
Dell inspiron 1100 battery
Dell 310-6321 battery
Dell 1691p battery
Dell Inspiron 500m battery
Dell 6Y270 battery
Dell inspiron 8600 battery
Latitude x300 series battery
Dell latitude cpi battery
Dell 1x793 battery
dell Inspiron 1501 battery
Dell 75UYF Battery
Dell Inspiron 1720 battery
dell Latitude C640 battery
Dell XPS M140 battery
Dell Inspiron E1405 battery
dell 700m battery
dell C1295 battery
Dell U4873 Battery
Dell Latitude C600 battery
Armada E700 Series battery
Compaq 116314-001 battery
Compaq 319411-001 battery
Compaq nc4200 battery
Compaq Presario R3000 Battery
Compaq Presario 2100 battery
Compaq Presario r3000 Battery
Compaq Business Notebook NX9000 series battery
HP 395789-001 battery
HP 446506-001 Battery

DiSCo said...

Really trustworthy blog. Please keep updating with great posts like this one. I have booked marked your site and am about to email it

to a few friends of mine that I know would enjoy reading..
seslisohbet
seslichat
sesli sohbet
sesli chat
sesli
sesli site
görünlütü sohbet
görüntülü chat
kameralı sohbet
kameralı chat
sesli sohbet siteleri
sesli chat siteleri
görüntülü sohbet siteleri
görüntülü chat siteleri
kameralı sohbet siteleri
canlı sohbet
sesli muhabbet
görüntülü muhabbet
kameralı muhabbet
seslidunya
seslisehir
sesli sex

Sesli Chat said...

Really trustworthy blog. Please keep updating with great posts like this one. I have booked marked your site and am about to email it

to a few friends of mine that I know would enjoy reading..
seslisohbet
seslichat
sesli sohbet
sesli chat
sesli
sesli site
görünlütü sohbet
görüntülü chat
kameralı sohbet
kameralı chat
sesli sohbet siteleri
sesli chat siteleri
sesli muhabbet siteleri
görüntülü sohbet siteleri
görüntülü chat siteleri
görüntülü muhabbet siteleri
kameralı sohbet siteleri
kameralı chat siteleri
kameralı muhabbet siteleri
canlı sohbet
sesli muhabbet
görüntülü muhabbet
kameralı muhabbet
birsesver
birses
seslidunya
seslisehir
sesli sex

Anonymous said...

коттедж
восстановление зрения
зеленый лазер
электрошокер

Anonymous said...

коттедж
восстановление зрения
зеленый лазер
электрошокер

Anonymous said...

коттедж
восстановление зрения
зеленый лазер
электрошокер

Anonymous said...

коттедж
восстановление зрения
зеленый лазер
электрошокер