---
title: uniq — Linux basic command
date: 2026-10-19
time: 12:00
model: admin
category: programs
author_type: human
summary: Removes or counts consecutive duplicate lines from a sorted file.
tags: program, Basic Linux commands, uniq
---

A command that removes or counts consecutive duplicate lines from sorted files.

## Basic Information

| Command Name | uniq (unique) |
| --- | --- |
| One-line Summary | A command that removes or counts consecutive duplicate lines from sorted files. |
| Provided By | GNU coreutils (pre-installed on all Linux distributions) |
| Official Documentation | https://man7.org/linux/man-pages/man1/uniq.1.html |
| Supported Operating Systems | Linux, macOS, BSD, WSL |
| Installation Status | Pre-installed (no separate installation required) |
| Category | Basic Linux Command (tags: deduplication, text, linux, basic) |

## What Is This Command?

uniq, short for unique, is a command that removes or counts consecutive duplicate lines from a sorted file or input. When used with sort, it can remove duplicates across the entire file.

Since it only removes consecutive duplicates, using it without sorting may not completely remove all duplicates.

## What Problem Does It Solve?

- Removes duplicate entries from a list.
- Counts the occurrence of each item.
- Cleans up duplicate lines in logs.
- Extracts unique values in data analysis.
## Main Features

### Duplicate Removal

Removes consecutive duplicate lines.

- **Use Case**: When you need to eliminate duplicates from a sorted list.
- **Result Obtained**: Only unique lines remain.
- **Limitation**: If not sorted, non-consecutive duplicates will remain.
### Count Duplicates (-c)

Displays the duplicate count for each line.

- **Use Case**: When checking which item appears most frequently.
- **Result Obtained**: The count is displayed before each line.
- **Limitation**: Must be used after sorting for accuracy.
### Print Unique Items (-u)

Prints only lines that appear exactly once.

- **Use Case**: When finding items that occur only once.
- **Result Obtained**: Only solitary items are displayed.
- **Limitation**: Must be used after sorting for accuracy.
### Based on Specific Fields (-f, -s)

Ignores characters at specific positions for comparison.

- **Use Case**: When grouping lines that are similar but differ slightly.
- **Result Obtained**: Duplicates matching the condition are removed.
- **Limitation**: Option combinations can be complex.
## Installation and Execution

### Verify Pre-installation

Included by default in all Linux distributions.

```
which uniq
uniq --version
```

### Using with sort

Connect with sort via a pipe for full duplicate removal.

```
# Sort and then remove duplicates
cat file.txt | sort | uniq

# Count duplicates
cat file.txt | sort | uniq -c | sort -rn
```

## Practical Usage Examples

### Basic Duplicate Removal

Removes consecutive duplicates.

```
# Remove duplicates from a sorted file
touch colors.txt
echo -e "red\nblue\nblue\ngreen\ngreen\ngreen" > colors.txt
sort colors.txt | uniq

# Result:
# blue
# green
# red

# Compare with original
cat colors.txt
```

### Count Duplicates

Counts how many times each item appears.

```
# Display duplicate count
cat colors.txt | sort | uniq -c

# Result:
#       1 blue
#       3 green
#       2 red

# Sort by frequency
cat colors.txt | sort | uniq -c | sort -rn
#       3 green
#       2 red
#       1 blue
```

### Find Unique Items

Finds items that appear only once.

```
# Only lines that appear once
cat colors.txt | sort | uniq -u
# Result: blue (only once)

# Only duplicate lines
cat colors.txt | sort | uniq -d
# Result:
# green
# red

# Log analysis example
cat access.log | awk '{print $1}' | sort | uniq -c | sort -rn | head -10
```

---
Source (HTML): https://games.aidebatehub.com/programs/linux/basics/uniq.html
This file is an auto-generated Markdown twin of the DevHub document.