[Linear Algebra Series Part 2] Seeing Scalars and Vectors Through a Programmer's Lens

What this post covers

This post separates the two smallest building blocks of linear algebra: scalars and vectors.

What makes a scalar different from a vector
Why vectors should be understood both as arrows and as coordinate representations
How coordinates, RGB values, feature vectors, and embeddings fit the vector idea
When a list of numbers should count as a real vector and when it is only a list of numbers

Key terms

scalar: a quantity represented by a single number
vector: an object whose components live in one shared space where addition and scaling make sense
basis: a minimal set of directions used later to describe a vector space
dimension: how many basis directions are needed to describe that space

Core idea

A scalar is a single number. Length, temperature, probability, and loss are all examples of scalar quantities. A vector, by contrast, is an object made of multiple numbers whose order matters and whose components sit on meaningful axes.

For example, 5 is a scalar. But (3, 4) can be a vector. The key point is not merely that it has two numbers. The more important point is that those numbers must stand for different axes inside the same world. By “the same world,” we mean that the components belong to one shared space where addition and scalar multiplication are meaningful.

(x, y) coordinates naturally form a vector because both values live in the same geometric space.
(r, g, b) can be treated as a vector because the three values refer to color channels.
A user feature tuple such as (click count, dwell time, purchase count) can become a vector when its axes are defined clearly.

By contrast, a bundle such as (height 175 cm, name length 3, favorite color blue) does not immediately behave like a vector in linear algebra. Having numbers is not enough. A vector must be the kind of object for which addition and scalar multiplication make coherent sense.

A vector is both an arrow and a coordinate representation

Introductory linear algebra often draws vectors as arrows. That picture is extremely useful for direction and magnitude intuition. Programmers, however, often read vectors as arrays or tuples. These two views do not compete. They complement each other.

The arrow view gives intuition for direction and size.
The coordinate view is what we actually compute with in code.

More precisely, a vector is an element of a vector space, and a notation like (3, 4) is a coordinate representation of that vector relative to some basis. Unless we say otherwise, we usually assume the standard basis such as (1, 0) and (0, 1) in 2D. You do not need to hold the full formalism tightly yet, but it is worth remembering that the same vector can have different coordinates if the basis changes.

What does an axis mean?

An axis is not just a slot number. It tells you what each component means.

In a coordinate vector, axes are geometric directions.
In a feature vector, axes are measured attributes such as clicks, time, or purchases.
In an embedding, an axis may be a latent feature that has no easy human name.

So an axis is closer to “what does this component mean?” than to “what index is this?”

Why the arrow-only view is not enough

Most data programmers work with is far beyond three dimensions. Sentence embeddings may have hundreds of dimensions, and an image can be viewed as a vector with thousands or millions of components. In those settings, the idea of a vector as an ordered representation of values placed on meaningful axes becomes more important than the picture of an arrow.

A useful practical summary is this:

vector = an ordered representation of same-kind features

That view will make later ideas such as span, basis, and dimension feel much more natural.

Step-by-step examples

Let us make the idea concrete with examples you have probably already seen in code or data work.

Example 1) Game coordinates

Suppose a character in a 2D game is at (10, 5). The first value is its x coordinate and the second is its y coordinate. Both values live in the same coordinate system, so vector operations make sense.

If the current position is (10, 5) and the displacement is (2, -1), then the new position is

(10, 5) + (2, -1) = (12, 4)

That calculation feels natural because both objects live on the same coordinate axes.

Example 2) RGB color

A color on the web can be written as (255, 120, 80). The three values stand for red, green, and blue channel intensity.

color = (255, 120, 80)

If you take the average of two colors, you are effectively averaging two vectors. In this post we will treat RGB as a vector in a simplified algebraic sense. Real color pipelines also involve gamma correction and perceptual color spaces, so the practical story is more complicated.

Example 3) A user feature vector

In recommender or analytics systems, we often describe a user with several numerical features.

user = (clicks in the last 7 days, average dwell time, purchase count)

Each coordinate now represents a different behavioral attribute. That makes it possible to compare users using distance, similarity, and clustering tools.

Example 4) Sentence embeddings

In natural language processing, a sentence may be mapped to a high-dimensional vector such as

sentence = (0.18, -1.02, 0.44, ..., 0.73)

The axes may not have names that humans can easily interpret. Still, as long as the representation is consistent inside the same space, the model can compare sentences meaningfully. In other words, a vector is still useful even when its axes are latent rather than explicitly labeled.

Example 5) From categories to vectors

A common beginner question is: “If a value is categorical or textual, can it ever become a vector?” Not in its raw form, usually. But it can be encoded into one.

One-hot encoding turns categories into 0/1 vectors. In fact, those one-hot vectors are the standard basis vectors of R^n.
Embeddings map categories or tokens into learned real-valued vectors.

So a better rule is not “categorical data is not a vector,” but rather raw categorical data is not immediately a vector, yet it can be vectorized through a suitable representation.

Why scale matters

Even if something can be represented as a vector, scale can still distort the story.

If one feature ranges from 0 to 100000 and another ranges from 0 to 1, the larger-scale feature may dominate distance calculations.
Standardization usually means rescaling each feature across a dataset, for example z = (x - mu) / sigma.
Normalization usually means rescaling one vector itself, for example v_normalized = v / ||v|| when ||v|| != 0.

Programmers use both, but for different reasons. Standardization helps different features contribute on comparable scales. Normalization changes the geometry of one vector, often to focus on direction rather than magnitude.

Math notes

The key idea is not the number of entries, but whether the object belongs to a space where the right operations are defined.
Saying that objects live in the same vector space means they can be added and scaled while remaining the same kind of object.
Scalars usually appear as the coefficients that control vectors. That is exactly what scalar multiplication will formalize next.
Ideas such as length, angle, and similarity become more precise only after we introduce norms and the dot product.
A short checklist helps: can you add two objects of this kind, scale them by real numbers, stay in the same space, and keep the usual algebraic rules?

Common notation uses a, b, c for scalars and x, v, w for vectors. For example, v = (v1, v2, v3) is a vector with three components.

Common mistakes

"Any list of numbers is a vector"

Not necessarily. For a numeric bundle to work as a vector, its axes must have a coherent meaning and the operations on it must make sense.

"Vectors are only arrows"

Some vectors are easy to draw as arrows, but many are not. Embeddings are usually too high-dimensional to visualize directly. The arrow picture is useful, but it is not the whole story.

"A tuple is the vector itself"

A tuple such as (3, 4) is often just a coordinate representation of a vector in a chosen basis. The vector is the underlying object; the tuple is one way to describe it.

Ignoring scale differences across axes

A feature vector may contain values on very different scales. That is why standardization and normalization exist, but they solve different problems and should not be confused.

Practice or extension

Decide whether it is natural to treat each of the following as a vector.

(x, y, z) position coordinates
(Korean score, English score, math score)
(name length, favorite color id, today's mood score)
(R, G, B)
A 768-dimensional sentence embedding
A one-hot encoded product category

Then ask yourself:

Do the axes represent the same kind of thing?
Does addition make sense?
Does scalar multiplication make sense?

That habit helps you distinguish data that only looks vector-like from data that should genuinely be treated as a vector.

Wrap-up

In this post, we separated scalars from vectors and tied vectors to programming-style data representation.

A scalar is a single number.
A vector is an ordered collection of values living on meaningful axes.
Coordinates, colors, feature vectors, and embeddings can all be treated as vectors.
But the meaning of the axes and their scale still matters.
Even categorical data can be vectorized once a suitable representation is chosen.

The next post introduces the two operations that everything else builds on: vector addition and scalar multiplication.