What else is wrong with Story Points as a time unit?

There is one another point against considering a story point as a certain period of time (e.g. half a day). It is difficult to use special planning cards for a planning session because their values are like 1, 2, 3, 5, 8, 13, 20, etc. but the time in other hand is linear. So the problem is what to do if we estimate that something should take 17 days?

The original post with comments

Why Story Points should be abstract units?

Imagine that you assumed that 1 story point is about 4 ideal hours of work and you made all the estimations according to that assumption. When the sprint finished you divided a sum of story points of done stories by a number of spent hours and the result was that 1 story point is 6 hours. So how to proceed during the next sprint planning? If we assume that 1 story point is now 6 hours, we estimate stories in a different way than before. In other hand, keeping in mind that 1 story point is 4 hours, although we know that last sprint it was 6 hours is also confusing. In the described case 1 story point corresponds to some certain time, so it looses its real meaning.

Story points should be some abstract units expressing relations between stories in a way a team asses them. When they are abstract there is no other way to estimate a story than by comparing it to already estimated stories. Thanks to that, estimating is always the same, there is no confusion.