book-reader/app/src/main/java/com/aryan/reader/epub/EpubParser.kt
Aryan b20ade9946
Desktop app (#308)
* Implement build profiles and feature policy for offline desktop builds

* Introduce unified cross-platform Settings Hub

* Refactor main settings into a hierarchical page-based navigation model

* Refactor library projection to use shared multiplatform logic

* Refactor UI state consumption by removing intermediate screen models

* Introduce AndroidSharedStateBridge to centralize state mapping and reduction logic

* Refactor state management for tabs, selection, and pinning to use shared bridge logic

* Refactor file type management and validation into a centralized shared module

* Centralize file type resolution and improve handling of unknown types

* Centralize book import logic with SharedImportPlanner

* Refactor magnifier geometry logic and coordinate mapping

* Properly handle orientation changes in scroll-locked PDF reader

* Add screen orientation controls to EPUB and PDF readers

* Implement right-to-left (RTL) pagination support and refactor reader menus

* Separate right-to-left pagination settings for PDF and EPUB

* Ensure PDF page data is scoped by document key for multi tab support

* Implement theme-aware link styling for the epub reader

* Implement jump history for back and forward navigation in the epub reader

* Improve locator handling and navigation logic in paginated reader mode

* Implement stable pagination navigation and location tracking

* Centralize banner message management and auto-dismiss logic in MainViewModel

* Implement zoom and pan state preservation for PDF pan lock mode

* Enhance reader navigation UI and workspace layout management in desktop app

* Refactor reader navigation sidebar and relocate search controls in desktop app

* Enhance reader UI with redesigned selection menus and bottom sheet overlays

* Implement custom highlight palettes and reader theme customization in desktop app

* Implement cross-platform modal layer and refine reader UI styling

* Improve highlight accuracy and implement metadata enrichment on book open in desktop app

* Implement two-page spread layout for paginated reader on desktop

* Implement persistent caching for book loading and pagination in desktop app

* Implement persistent caching for book loading and pagination in desktop app

* Optimize reader settings updates by separating layout and appearance changes in desktop app

* Improve desktop window branding and native Windows styling

* Enhance reader selection interactions and UI across EPUB and PDF viewers in desktop app

* Refine selection handle positioning and interaction logic

* Implement EPUB selection debug logging and improve handle targeting

* Optimize desktop book loading performance and UI responsiveness

* Implement anchored zoom gestures and rendering optimizations for the Desktop PDF viewer.

* Implement smooth zoom preview for the PDF reader in desktop app

* Optimize PDF rendering performance and responsiveness in the desktop reader

* Implement conditional diagnostic logging and update desktop build configuration

* Implemented hierarchical TOC, custom scrollbars, and improved desktop modal handling

* Added management options for annotations and highlights in the sidebar in desktop app

* Implemented `SharedStableOutlinedTextField` and updated text input fields to use `TextFieldValue` for improved cursor and selection stability.

* Refined library filters and enhanced OPDS functionality in desktop app

* Improved EPUB pagination measurement and implemented layout diagnostic logging for desktop app

* Added PPTX support including document parsing, rendering, and indexing

* Improved PPTX rendering and layout accuracy

* Implemented text autofit support for PPTX rendering

* Enhanced PPTX rendering with support for custom geometry, automatic numbering, table styles, and image opacity

* Improved EPUB pagination accuracy and added layout telemetry in desktop app

* Improved folder synchronization with metadata-only mode and hashed sidecar management in desktop app

* Implemented rich text font scaling and migrated desktop ink tools to custom pointer input handling

* Implemented billing account obfuscation

* Implemented hierarchical folder navigation and improved library selection functionality in desktop app

* Implemented platform-aware directory resolution and multi-platform native library support for desktop

* Added full-screen mode for the reader workspace

* Added PDF zoom indicator and interactive vertical scrollbar with page tooltips

* Refactored speech bubble prefetching to use a limited radius and improved ML detector initialization and lifecycle management

* Updated PDF indexing to replace existing page text and removed search result item keys

* Implemented "preparing" foreground notification for TTS service

* Optimized PDF rendering performance by pre-calculating page-specific annotations

* Refactored desktop packaging tasks and improved distribution configuration

* Optimized EPUB parser memory usage and added path traversal protection

* Refactored WorkManager monitoring logic and added work pruning

* Implemented comprehensive resource cleanup and memory management for WebView-based components to prevent memory leaks

* Implemented bitmap size limits and scaling to prevent canvas rendering errors

* Split long text paragraphs into multiple semantic blocks during HTML parsing

* Implemented local ActionMode for text selection to prevent platform crashes

* Refactored PPTX text layout, optimized HtmlParser block detection, and improved banner dismissal logic

* Added desktop startup splash screen and deferred WebView initialization

* Reorganized settings hub and added separate PDF reader defaults

* Implemented embedded cover extraction and metadata support for MOBI and FB2 formats

* Implemented batching for MetadataExtractionWorker and optimized EPUB metadata extraction performance.

* Implemented procedurally generated book covers and replaced static placeholders

* Redesigned search UI with a top bar and results overlay in desktop app

* Added PDF page gap and overlay visibility options and implemented DesktopBookImporter

* Refactored PDF reader UI with tabbed inspector and improved theme background handling in desktop

* Implemented PDF viewport persistence for zoom and scroll positions in desktop app

* Improved desktop fullscreen implementation and state restoration

* Implemented desktop window state persistence

* Implemented flavor-based branding and ProGuard configuration for desktop builds

* Implemented precise reader positioning and improved highlight rendering logic in desktop app

* Added support for user-editable book metadata

* Enhanced book metadata support and integrated info/edit dialogs

* Implemented embedded EPUB metadata editing

* Improved highlight mapping and added custom scrollbar styling for the reader.

* Reduced desktop WebView bundle size by excluding unused locales and runtime files

* Added neutral pan mode as the default PDF interaction state.

* Refactored library empty states and updated primary navigation tabs in desktop app

* Implemented native paginated reader and unified content rendering architecture in desktop epub reader

* Implemented native EPUB image rendering for desktop and improved block layout spacing with margin collapsing.

* Improved pagination overflow detection in desktop

* Implemented multi-block text selection with interactive handles and CFI support in desktop epub pagination
2026-05-15 22:36:51 +05:30

938 lines
39 KiB
Kotlin

/*
* Episteme Reader - A native Android document reader.
* Copyright (C) 2026 Episteme
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as
* published by the Free Software Foundation, either version 3 of the
* License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*
* mail: epistemereader@gmail.com
*/
package com.aryan.reader.epub
import android.content.Context
import android.graphics.Bitmap
import android.graphics.BitmapFactory
import timber.log.Timber
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
import kotlinx.serialization.Serializable
import kotlinx.serialization.decodeFromString
import kotlinx.serialization.encodeToString
import kotlinx.serialization.json.Json
import org.jsoup.Jsoup
import org.w3c.dom.Element
import org.w3c.dom.Node
import java.io.File
import java.io.FileOutputStream
import java.io.InputStream
import java.io.ByteArrayOutputStream
import java.net.URLDecoder
import java.nio.file.Paths
import java.util.UUID
import java.util.zip.ZipEntry
import java.util.zip.ZipFile
import kotlinx.coroutines.async
import kotlinx.coroutines.awaitAll
import kotlinx.coroutines.sync.Semaphore
import kotlinx.coroutines.sync.withPermit
class EpubParser(private val context: Context) {
private val jsonSerializer = Json { ignoreUnknownKeys = true; encodeDefaults = true }
data class EpubDocument(
val metadata: Node, val manifest: Node, val spine: Node, val opfFilePath: String
)
data class EpubManifestItem(
val id: String, val absPath: String, val mediaType: String, val properties: String
)
data class TempEpubChapter(
val url: String,
val title: String?,
val htmlFilePath: String,
val chapterIndex: Int,
val plainTextContent: String,
val htmlContent: String,
val depth: Int,
val isInToc: Boolean
)
// Helper class for NCX parsing results
data class NcxMetadata(
val title: String,
val depth: Int
)
@Serializable
private data class EpubExtractionCacheManifest(
val bookId: String,
val originalBookNameHint: String,
val parserVersion: Int,
val parseContent: Boolean,
val shouldUseToc: Boolean,
val sourceFingerprint: String? = null
)
// EpubFile can still represent in-memory file data during initial parsing before extraction
data class EpubFile(val absPath: String, val data: ByteArray) {
override fun equals(other: Any?): Boolean {
if (this === other) return true
if (javaClass != other?.javaClass) return false
other as EpubFile
if (absPath != other.absPath) return false
return data.contentEquals(other.data)
}
override fun hashCode(): Int {
var result = absPath.hashCode()
result = 31 * result + data.contentHashCode()
return result
}
}
@Serializable
data class EpubPageTarget(
val id: String?,
val value: String?,
val label: String?,
val contentSrc: String
)
companion object {
const val TAG = "EpubParser"
private const val BOOK_METADATA_FILE = "book_metadata.json"
private const val CACHE_MANIFEST_FILE = "epub_cache_manifest.json"
private const val EPUB_EXTRACTION_CACHE_VERSION = 1
private const val MAX_METADATA_ENTRY_BYTES = 4 * 1024 * 1024
private const val EPUB_COVER_MAX_DIMENSION = 1024
private val EPUB_IMAGE_EXTENSIONS = setOf(".png", ".jpg", ".jpeg", ".gif", ".webp", ".svg")
}
internal val String.decodedURL: String
get() = try {
URLDecoder.decode(this, "UTF-8")
} catch (e: Exception) {
Timber.w(e, "Failed to decode URL: $this")
this
}
private fun parsePageList(pageListElement: Element?, ncxFileParentDir: File): List<EpubPageTarget> {
if (pageListElement == null) {
Timber.d("No <pageList> element found in NCX.")
return emptyList()
}
val pageTargets = mutableListOf<EpubPageTarget>()
pageListElement.selectChildTag("pageTarget").forEach { ptElement ->
val contentSrcRaw = ptElement.selectFirstChildTag("content")?.getAttributeValue("src")?.decodedURL
if (contentSrcRaw != null) {
val contentPathRelativeToEpubRoot = Paths.get(ncxFileParentDir.path, contentSrcRaw)
.normalize().toString().replace(File.separatorChar, '/')
pageTargets.add(
EpubPageTarget(
id = ptElement.getAttributeValue("id"),
value = ptElement.getAttributeValue("value"),
label = ptElement.selectFirstChildTag("navLabel")?.selectFirstChildTag("text")?.textContent,
contentSrc = contentPathRelativeToEpubRoot
)
)
} else {
Timber.w("PageTarget found with no content src: ${ptElement.getAttributeValue("id")}")
}
}
Timber.d("Parsed ${pageTargets.size} page targets from NCX.")
return pageTargets
}
private fun parseEpubCss(
manifestItems: Map<String, EpubManifestItem>,
filesContentMap: Map<String, EpubFile>,
extractionRoot: File
): Map<String, String> {
val listedCss = manifestItems.values
.filter { it.mediaType == "text/css" }
.mapNotNull { manifestItem ->
val bytes = filesContentMap[manifestItem.absPath]?.data?.takeIf { it.isNotEmpty() }
?: File(extractionRoot, manifestItem.absPath).takeIf { it.exists() }?.readBytes()
bytes?.let { manifestItem.absPath to String(it, Charsets.UTF_8) }
}
val listedCssPaths = listedCss.map { it.first }.toSet()
val unlistedCss = filesContentMap.asSequence()
.filter { (path, _) -> path.endsWith(".css", ignoreCase = true) }
.filterNot { (path, _) -> listedCssPaths.contains(path) }
.mapNotNull { (path, file) ->
val bytes = file.data.takeIf { it.isNotEmpty() }
?: File(extractionRoot, path).takeIf { it.exists() }?.readBytes()
bytes?.let { path to String(it, Charsets.UTF_8) }
}
.toList()
val allCss = (listedCss + unlistedCss).toMap()
Timber.d("Parsed ${allCss.size} CSS files (listed: ${listedCss.size}, unlisted: ${unlistedCss.size}): ${allCss.keys.joinToString()}")
return allCss
}
suspend fun createEpubBook(
inputStream: InputStream,
bookId: String,
shouldUseToc: Boolean = true,
originalBookNameHint: String = "streamed_book",
parseContent: Boolean = true,
extractionDirOverride: File? = null,
sourceFingerprint: String? = null
): EpubBook {
return withContext(Dispatchers.IO) {
Timber.d("Parsing EPUB input stream for bookId: $bookId")
val shouldDeleteExtractionDir = !parseContent && extractionDirOverride == null
val extractionDir = if (extractionDirOverride != null) {
ImportedFileCache.prepareDirectory(extractionDirOverride)
} else if (!parseContent) {
ImportedFileCache.createTemporaryBookDir(context, bookId, "metadata")
} else {
val activeDir = ImportedFileCache.ensureActiveBookDir(context, bookId)
readCachedEpubBook(
extractionDir = activeDir,
bookId = bookId,
originalBookNameHint = originalBookNameHint,
shouldUseToc = shouldUseToc,
sourceFingerprint = sourceFingerprint
)?.let { cachedBook ->
Timber.tag("FileOpenPerf").d("[EPUB] Loaded extracted book from cache | bookId=$bookId")
return@withContext cachedBook
}
ImportedFileCache.resetActiveBookDir(context, bookId)
}
val tempFile = File.createTempFile("epub_stream", ".epub", context.cacheDir)
val filesMap: Map<String, EpubFile>
try {
tempFile.outputStream().use { output ->
inputStream.copyTo(output)
}
filesMap = extractEpubContents(
zipFile = ZipFile(tempFile),
extractionDir = extractionDir,
parseContent = parseContent,
extractImagesForMetadata = shouldDeleteExtractionDir
)
} finally {
tempFile.delete()
}
val document = createEpubDocument(filesMap)
val book = parseAndCreateEbook(filesMap, document, shouldUseToc, extractionDir.absolutePath,
originalBookNameHint, parseContent)
if (parseContent && extractionDirOverride == null) {
writeCachedEpubBook(
extractionDir = extractionDir,
bookId = bookId,
originalBookNameHint = originalBookNameHint,
shouldUseToc = shouldUseToc,
sourceFingerprint = sourceFingerprint,
book = book
)
}
if (shouldDeleteExtractionDir) {
extractionDir.deleteRecursively()
}
return@withContext book
}
}
private fun readCachedEpubBook(
extractionDir: File,
bookId: String,
originalBookNameHint: String,
shouldUseToc: Boolean,
sourceFingerprint: String?
): EpubBook? {
val metadataFile = File(extractionDir, BOOK_METADATA_FILE)
val manifestFile = File(extractionDir, CACHE_MANIFEST_FILE)
if (!metadataFile.isFile || !manifestFile.isFile) return null
return try {
val manifest = jsonSerializer.decodeFromString<EpubExtractionCacheManifest>(manifestFile.readText())
val isCompatible = manifest.bookId == bookId &&
manifest.originalBookNameHint == originalBookNameHint &&
manifest.parserVersion == EPUB_EXTRACTION_CACHE_VERSION &&
manifest.parseContent &&
manifest.shouldUseToc == shouldUseToc &&
manifest.sourceFingerprint == sourceFingerprint
if (!isCompatible) {
Timber.d("EPUB extraction cache manifest is stale for bookId=$bookId")
return null
}
val cachedBook = jsonSerializer.decodeFromString<EpubBook>(metadataFile.readText())
.copy(extractionBasePath = extractionDir.absolutePath)
cachedBook.takeIf { it.hasReadableExtractedContent() }
} catch (e: Exception) {
Timber.e(e, "Failed to read EPUB extraction cache for bookId=$bookId")
null
}
}
private fun writeCachedEpubBook(
extractionDir: File,
bookId: String,
originalBookNameHint: String,
shouldUseToc: Boolean,
sourceFingerprint: String?,
book: EpubBook
) {
try {
File(extractionDir, BOOK_METADATA_FILE).writeText(jsonSerializer.encodeToString(book))
File(extractionDir, CACHE_MANIFEST_FILE).writeText(
jsonSerializer.encodeToString(
EpubExtractionCacheManifest(
bookId = bookId,
originalBookNameHint = originalBookNameHint,
parserVersion = EPUB_EXTRACTION_CACHE_VERSION,
parseContent = true,
shouldUseToc = shouldUseToc,
sourceFingerprint = sourceFingerprint
)
)
)
} catch (e: Exception) {
Timber.e(e, "Failed to write EPUB extraction cache for bookId=$bookId")
}
}
internal fun extractEpubContents(
zipFile: ZipFile,
extractionDir: File,
parseContent: Boolean,
extractImagesForMetadata: Boolean
): Map<String, EpubFile> {
val filesMap = mutableMapOf<String, EpubFile>()
zipFile.use { zf ->
zf.entries().asSequence().filterNot { it.isDirectory }.forEach { entry ->
val isEssential = isEssentialFile(entry.name, parseContent)
val isImage = isEpubImageFile(entry.name)
if (!parseContent) {
when {
isEssential -> {
val data = zf.readSmallEntryBytes(entry) ?: return@forEach
filesMap[entry.name] = EpubFile(absPath = entry.name, data = data)
}
isImage && extractImagesForMetadata -> {
val outputFile = safeExtractionFile(extractionDir, entry.name)
?: return@forEach
outputFile.parentFile?.mkdirs()
zf.getInputStream(entry).use { input ->
FileOutputStream(outputFile).use { output ->
input.copyTo(output)
}
}
filesMap[entry.name] = EpubFile(absPath = entry.name, data = ByteArray(0))
}
}
return@forEach
}
val outputFile = safeExtractionFile(extractionDir, entry.name) ?: return@forEach
outputFile.parentFile?.mkdirs()
zf.getInputStream(entry).use { input ->
FileOutputStream(outputFile).use { output ->
input.copyTo(output)
}
}
val lowerName = entry.name.lowercase()
val isContainerOrOpf = lowerName.endsWith("container.xml") || lowerName.endsWith(".opf")
val data = if (isContainerOrOpf) outputFile.readBytes() else ByteArray(0)
filesMap[entry.name] = EpubFile(absPath = entry.name, data = data)
}
}
return filesMap
}
private fun isEpubImageFile(fileName: String): Boolean {
val lowerName = fileName.lowercase()
return EPUB_IMAGE_EXTENSIONS.any { lowerName.endsWith(it) }
}
private fun safeExtractionFile(extractionDir: File, entryName: String): File? {
val outputFile = File(extractionDir, entryName)
val root = extractionDir.canonicalFile
val target = outputFile.canonicalFile
val rootPath = root.path
val targetPath = target.path
val isInsideRoot = targetPath == rootPath || targetPath.startsWith(rootPath + File.separator)
if (!isInsideRoot) {
Timber.w("Skipping unsafe EPUB entry outside extraction root: $entryName")
return null
}
return outputFile
}
private fun ZipFile.readSmallEntryBytes(entry: ZipEntry): ByteArray? {
if (entry.size > MAX_METADATA_ENTRY_BYTES.toLong()) {
Timber.w("Skipping oversized EPUB metadata entry: ${entry.name} (${entry.size} bytes)")
return null
}
val initialSize = entry.size
.takeIf { it in 0..MAX_METADATA_ENTRY_BYTES.toLong() }
?.toInt()
?: DEFAULT_BUFFER_SIZE
val buffer = ByteArray(DEFAULT_BUFFER_SIZE)
var totalBytes = 0
return getInputStream(entry).use { input ->
ByteArrayOutputStream(initialSize).use { output ->
while (true) {
val read = input.read(buffer)
if (read == -1) break
totalBytes += read
if (totalBytes > MAX_METADATA_ENTRY_BYTES) {
Timber.w("Skipping oversized EPUB metadata entry while reading: ${entry.name}")
return null
}
output.write(buffer, 0, read)
}
output.toByteArray()
}
}
}
private suspend fun parseAndCreateEbook(
filesContentMap: Map<String, EpubFile>,
document: EpubDocument,
shouldUseToc: Boolean,
extractionBasePath: String,
originalFilePathOrKey: String,
parseContent: Boolean = true
): EpubBook = withContext(Dispatchers.IO) {
val metadataTitle =
document.metadata.selectFirstChildTag("dc:title")?.textContent ?: File(originalFilePathOrKey).nameWithoutExtension
val metadataAuthor =
document.metadata.selectFirstChildTag("dc:creator")?.textContent ?: "Unknown Author"
val metadataLanguage =
document.metadata.selectFirstChildTag("dc:language")?.textContent ?: "en"
val metadataCoverId = getMetadataCoverId(document.metadata)
val metadataDescription =
document.metadata.selectFirstChildTag("dc:description")?.textContent
Timber.d("EpubParser: Extracted OPF metadata: title='$metadataTitle', author='$metadataAuthor'")
var metadataSeriesName: String? = null
var metadataSeriesIndex: Double? = null
document.metadata.selectChildTag("meta")
.ifEmpty { document.metadata.selectChildTag("opf:meta") }
.forEach { meta ->
val nameAttr = meta.getAttributeValue("name")
val contentAttr = meta.getAttributeValue("content")
if (nameAttr == "calibre:series") metadataSeriesName = contentAttr
if (nameAttr == "calibre:series_index") metadataSeriesIndex = contentAttr?.toDoubleOrNull()
}
val opfRelativePath = document.opfFilePath
val opfParentDir = File(opfRelativePath).parentFile ?: File("")
val manifestItems = getManifestItems(document.manifest, opfParentDir)
var pageTargets: List<EpubPageTarget> = emptyList()
val ncxMetadataMap = mutableMapOf<String, NcxMetadata>()
val extractionRoot = File(extractionBasePath)
if (shouldUseToc) {
Timber.d("shouldUseToc is true. Attempting to parse NCX.")
val tocFileItem = manifestItems.values.firstOrNull {
it.absPath.endsWith(".ncx", ignoreCase = true)
}
if (tocFileItem != null) {
val ncxParentDir = File(tocFileItem.absPath).parentFile ?: File("")
val ncxData = filesContentMap[tocFileItem.absPath]?.data?.takeIf { it.isNotEmpty() }
?: File(extractionRoot, tocFileItem.absPath).takeIf { it.exists() }?.readBytes()
val tocDocumentNode = ncxData?.let { parseXMLFile(it) }
if (tocDocumentNode != null) {
Timber.d("Successfully parsed NCX file: ${tocFileItem.absPath}")
val pageListElement = tocDocumentNode.selectFirstTag("pageList") as Element?
pageTargets = parsePageList(pageListElement, ncxParentDir)
val navMapElement = tocDocumentNode.selectFirstTag("navMap") as Element?
if (navMapElement != null) {
// Recursively parse navMap
ncxMetadataMap.putAll(parseNavMapRecursive(navMapElement, ncxParentDir))
} else {
Timber.d("No <navMap> element found in NCX.")
}
} else {
Timber.w("NCX file item '${tocFileItem.absPath}' found in manifest but could not be parsed.")
}
} else {
Timber.d("No NCX file found in manifest. Skipping NCX-based PageList/NavMap.")
}
} else {
Timber.d("shouldUseToc is false. Skipping NCX parsing for PageList/NavMap.")
}
Timber.d("Parsing chapters based on OPF spine for rendering order. NCX titles/depth will be used if available.")
val tableOfContents = if (shouldUseToc) {
val tocFileItem = manifestItems.values.firstOrNull {
it.absPath.endsWith(".ncx", ignoreCase = true)
}
if (tocFileItem != null) {
val ncxParentDir = File(tocFileItem.absPath).parentFile ?: File("")
val ncxData = filesContentMap[tocFileItem.absPath]?.data?.takeIf { it.isNotEmpty() }
?: File(extractionRoot, tocFileItem.absPath).takeIf { it.exists() }?.readBytes()
val tocDocumentNode = ncxData?.let { parseXMLFile(it) }
val navMapElement = tocDocumentNode?.selectFirstTag("navMap") as Element?
if (navMapElement != null) {
parseTableOfContents(navMapElement, ncxParentDir)
} else {
emptyList()
}
} else {
emptyList()
}
} else {
emptyList()
}
val chaptersFromSpine = if (parseContent) {
parseUsingSpine(document.spine, manifestItems, filesContentMap, ncxMetadataMap, extractionRoot)
} else {
emptyList()
}
Timber.d("Parsing images (for cover and general access)")
val images = if (parseContent) {
parseEpubImages(manifestItems, filesContentMap, extractionRoot)
} else {
emptyList()
}
Timber.d("Parsing cover image")
val coverImage = parseCoverImage(metadataCoverId, manifestItems, filesContentMap, extractionRoot)
val cssContent = if (parseContent) {
parseEpubCss(manifestItems, filesContentMap, extractionRoot)
} else {
emptyMap()
}
Timber.d("EpubBook created with ${chaptersFromSpine.size} spine chapters.")
return@withContext EpubBook(
fileName = metadataTitle.asFileName(),
title = metadataTitle,
author = metadataAuthor,
language = metadataLanguage,
coverImage = coverImage,
chapters = chaptersFromSpine, chaptersForPagination = chaptersFromSpine,
images = images,
pageList = pageTargets,
tableOfContents = tableOfContents,
extractionBasePath = extractionBasePath,
css = cssContent,
seriesName = metadataSeriesName,
seriesIndex = metadataSeriesIndex,
description = metadataDescription
)
}
private fun parseTableOfContents(
navMapElement: Element,
ncxParentDir: File
): List<EpubTocEntry> {
val result = mutableListOf<EpubTocEntry>()
fun recurse(element: Element, currentDepth: Int) {
val navPoints = element.childElements.filter { it.tagName == "navPoint" }
for (navPoint in navPoints) {
val label = navPoint.selectFirstChildTag("navLabel")
?.selectFirstChildTag("text")?.textContent?.trim() ?: "Untitled"
val contentSrc = navPoint.selectFirstChildTag("content")
?.getAttributeValue("src")?.decodedURL
if (contentSrc != null) {
val fullPathRaw = Paths.get(ncxParentDir.path, contentSrc)
.normalize().toString().replace(File.separatorChar, '/')
val parts = fullPathRaw.split("#", limit = 2)
val absolutePath = parts[0]
val fragmentId = if (parts.size > 1) parts[1] else null
result.add(EpubTocEntry(label, absolutePath, fragmentId, currentDepth))
recurse(navPoint, currentDepth + 1)
}
}
}
recurse(navMapElement, 0)
return result
}
@Throws(EpubParserException::class)
private fun createEpubDocument(files: Map<String, EpubFile>): EpubDocument {
val containerFile = files["META-INF/container.xml"]
?: throw EpubParserException("META-INF/container.xml file missing")
val rawOpfPath = parseXMLFile(containerFile.data)?.selectFirstTag("rootfile")
?.getAttributeValue("full-path")?.decodedURL
?: throw EpubParserException("Invalid container.xml: Could not find rootfile full-path.")
val opfFilePath = rawOpfPath.trimStart('/')
val opfFile = files[opfFilePath]
?: throw EpubParserException(".opf file missing at normalized path '$opfFilePath'.")
val document = parseXMLFile(opfFile.data)
?: throw EpubParserException(".opf file failed to parse data from '$opfFilePath'")
val metadata = document.selectFirstTag("metadata")
?: document.selectFirstTag("opf:metadata")
?: throw EpubParserException(".opf file metadata section missing in '$opfFilePath'")
val manifest = document.selectFirstTag("manifest")
?: document.selectFirstTag("opf:manifest")
?: throw EpubParserException(".opf file manifest section missing in '$opfFilePath'")
val spine = document.selectFirstTag("spine")
?: document.selectFirstTag("opf:spine")
?: throw EpubParserException(".opf file spine section missing in '$opfFilePath'")
return EpubDocument(metadata, manifest, spine, opfFilePath)
}
private fun getMetadataCoverId(metadata: Node): String? {
return metadata.selectChildTag("meta")
.ifEmpty { metadata.selectChildTag("opf:meta") }
.find { it.getAttributeValue("name") == "cover" }?.getAttributeValue("content")
}
private fun getManifestItems(
manifest: Node,
opfParentDir: File
): Map<String, EpubManifestItem> {
return manifest.selectChildTag("item")
.ifEmpty { manifest.selectChildTag("opf:item") }
.mapNotNull { itemElement ->
val href = itemElement.getAttribute("href")?.decodedURL ?: return@mapNotNull null
val pathRelativeToEpubRoot = Paths.get(opfParentDir.path, href)
.normalize().toString().replace(File.separatorChar, '/')
EpubManifestItem(
id = itemElement.getAttribute("id"),
absPath = pathRelativeToEpubRoot,
mediaType = itemElement.getAttribute("media-type"),
properties = itemElement.getAttribute("properties")
)
}.associateBy { it.id }
}
private fun parseNavMapRecursive(
element: Element,
ncxFileParentDir: File,
depth: Int = 0
): Map<String, NcxMetadata> {
val result = mutableMapOf<String, NcxMetadata>()
val navPoints = element.childElements.filter { it.tagName == "navPoint" }
for (navPoint in navPoints) {
val navLabelText = navPoint.selectFirstChildTag("navLabel")
?.selectFirstChildTag("text")?.textContent?.trim()
val contentSrcRaw = navPoint.selectFirstChildTag("content")
?.getAttributeValue("src")?.decodedURL
if (navLabelText != null && contentSrcRaw != null && navLabelText.isNotEmpty()) {
val contentPathRelativeToEpubRoot = Paths.get(ncxFileParentDir.path, contentSrcRaw)
.normalize().toString().replace(File.separatorChar, '/')
.substringBefore('#')
if (!result.containsKey(contentPathRelativeToEpubRoot)) {
result[contentPathRelativeToEpubRoot] = NcxMetadata(navLabelText, depth)
Timber.d("NCX Map: '$contentPathRelativeToEpubRoot' -> '$navLabelText' (Depth $depth)")
}
}
result.putAll(parseNavMapRecursive(navPoint, ncxFileParentDir, depth + 1))
}
return result
}
private fun generateId(): String {
return UUID.randomUUID().toString()
}
private suspend fun parseUsingSpine(
spine: Node,
manifestItems: Map<String, EpubManifestItem>,
filesContentMap: Map<String, EpubFile>,
ncxMetadataMap: Map<String, NcxMetadata>,
extractionRoot: File
): List<EpubChapter> = withContext(Dispatchers.Default) {
val parsingSemaphore = Semaphore(6)
val spineItems = spine.selectChildTag("itemref")
.ifEmpty { spine.selectChildTag("opf:itemref") }
val deferredChapters = spineItems.mapIndexed { index, itemRef ->
async {
parsingSemaphore.withPermit {
val idRef = itemRef.getAttribute("idref")
val item = manifestItems[idRef] ?: return@withPermit null
val fileBytes = filesContentMap[item.absPath]?.data?.takeIf { it.isNotEmpty() }
?: File(extractionRoot, item.absPath).takeIf { it.exists() }?.readBytes()
?: return@withPermit null
val mediaType = item.mediaType
val absPath = item.absPath
if (mediaType.startsWith("application/xhtml+xml") ||
mediaType.startsWith("text/html") ||
absPath.endsWith(".html", ignoreCase = true) ||
absPath.endsWith(".xhtml", ignoreCase = true) ||
absPath.endsWith(".xml", ignoreCase = true)
) {
val rawHtml = String(fileBytes, Charsets.UTF_8)
val document = Jsoup.parse(rawHtml)
val plainText = document.text()
val parser = EpubXMLFileParser(
fileRelativePath = absPath,
data = fileBytes,
fragmentId = null
)
val res = parser.parseForTitleAndPath(document)
val chapterTitleFromHtml = res.title
val ncxKey = absPath.substringBefore('#')
val ncxData = ncxMetadataMap[ncxKey]
val isEffectiveInToc = if (ncxMetadataMap.isNotEmpty()) {
ncxData != null
} else {
true
}
val finalChapterTitle = if (ncxData != null && ncxData.title.isNotBlank()) {
ncxData.title
} else {
chapterTitleFromHtml
}
val finalDepth = ncxData?.depth ?: 0
TempEpubChapter(
url = absPath,
title = finalChapterTitle,
htmlFilePath = res.effectiveHtmlPath,
chapterIndex = index + 1,
plainTextContent = plainText,
htmlContent = "", // OPTIMIZATION: Don't store HTML in memory, it's on disk
depth = finalDepth,
isInToc = isEffectiveInToc
)
} else if (mediaType.startsWith("image/")) {
// Image handling remains similar, but usually small enough
val htmlContent = """
<!DOCTYPE html><html style="margin:0;padding:0;height:100%;"><head><title>Image</title></head><body style="margin:0;padding:0;height:100%;text-align:center;"><img src="$absPath" alt="Image from spine" style="object-fit:contain;width:100%;height:100%;"/></body></html>
""".trimIndent()
val ncxKey = absPath.substringBefore('#')
val ncxData = ncxMetadataMap[ncxKey]
val isEffectiveInToc = if (ncxMetadataMap.isNotEmpty()) ncxData != null else true
TempEpubChapter(
url = absPath,
title = ncxData?.title ?: "Image",
htmlFilePath = absPath,
chapterIndex = index + 1,
plainTextContent = "[Image]",
htmlContent = htmlContent,
depth = ncxData?.depth ?: 0,
isInToc = isEffectiveInToc
)
} else {
null
}
}
}
}
val tempChapters = deferredChapters.toList().awaitAll().filterNotNull()
return@withContext tempChapters.map { tempChapter ->
EpubChapter(
chapterId = generateId(),
absPath = tempChapter.url,
title = tempChapter.title?.takeIf { it.isNotBlank() } ?: "Chapter ${tempChapter.chapterIndex}",
htmlFilePath = tempChapter.htmlFilePath,
plainTextContent = tempChapter.plainTextContent,
htmlContent = tempChapter.htmlContent,
depth = tempChapter.depth,
isInToc = tempChapter.isInToc
)
}.filter { it.htmlFilePath.isNotBlank() }
}
private fun parseEpubImages(
manifestItems: Map<String, EpubManifestItem>,
filesContentMap: Map<String, EpubFile>,
@Suppress("UNUSED_PARAMETER") extractionRoot: File
): List<EpubImage> {
val listedImages = manifestItems.values
.filter { it.mediaType.startsWith("image/") }
.map { manifestItem ->
EpubImage(absPath = manifestItem.absPath)
}
val listedPaths = listedImages.map { it.absPath }.toSet()
val unlistedImages = filesContentMap.keys
.filter { path ->
val lowerPath = path.lowercase()
EPUB_IMAGE_EXTENSIONS.any { lowerPath.endsWith(it) } && !listedPaths.contains(path)
}
.map { path ->
EpubImage(absPath = path)
}
Timber.d("Identified ${listedImages.size + unlistedImages.size} images (content not loaded into memory).")
return (listedImages + unlistedImages).distinctBy { it.absPath }
}
private fun parseCoverImage(
metadataCoverId: String?,
manifestItems: Map<String, EpubManifestItem>,
filesContentMap: Map<String, EpubFile>,
extractionRoot: File
): Bitmap? {
val coverManifestItem = manifestItems[metadataCoverId]
if (coverManifestItem != null) {
val coverImage = decodeEpubImage(coverManifestItem.absPath, filesContentMap, extractionRoot)
if (coverImage != null) {
return coverImage
} else {
Timber.e("Cover image file content not found for path: ${coverManifestItem.absPath}")
}
} else {
if (metadataCoverId != null) {
Timber.w("Cover image ID '$metadataCoverId' not found in manifest.")
} else {
Timber.d("No cover image ID specified in metadata.")
}
}
val commonCoverNames = listOf("cover.jpg", "cover.jpeg", "cover.png")
for (name in commonCoverNames) {
val possiblePaths = listOf(
name, "images/$name", "Images/$name", "image/$name", "Image/$name",
"OEBPS/images/$name", "OEBPS/Images/$name", "OEBPS/image/$name", "OEBPS/Image/$name",
"OPS/images/$name", "OPS/Images/$name", "OPS/image/$name", "OPS/Image/$name"
)
for (path in possiblePaths) {
if (filesContentMap.containsKey(path)) {
decodeEpubImage(path, filesContentMap, extractionRoot)?.let {
Timber.d("Found fallback cover image at $path")
return it
}
}
manifestItems.values.find { item -> item.absPath.equals(path, ignoreCase = true) && item.mediaType.startsWith("image/") }?.let { manifestItem ->
decodeEpubImage(manifestItem.absPath, filesContentMap, extractionRoot)?.let {
Timber.d("Found fallback cover image via manifest item (case-insensitive) at ${manifestItem.absPath}")
return it
}
}
}
}
Timber.d("Cover image could not be loaded from metadata or common fallbacks.")
return null
}
private fun decodeEpubImage(
path: String,
filesContentMap: Map<String, EpubFile>,
extractionRoot: File
): Bitmap? {
filesContentMap[path]?.data?.takeIf { it.isNotEmpty() }?.let { bytes ->
return decodeSampledByteArray(bytes)
}
val imageFile = File(extractionRoot, path).takeIf { it.exists() && it.isFile } ?: return null
return decodeSampledFile(imageFile)
}
private fun decodeSampledByteArray(bytes: ByteArray): Bitmap? {
val bounds = BitmapFactory.Options().apply {
inJustDecodeBounds = true
}
BitmapFactory.decodeByteArray(bytes, 0, bytes.size, bounds)
val options = BitmapFactory.Options().apply {
inSampleSize = calculateBitmapSampleSize(bounds)
}
return BitmapFactory.decodeByteArray(bytes, 0, bytes.size, options)
}
private fun decodeSampledFile(file: File): Bitmap? {
val bounds = BitmapFactory.Options().apply {
inJustDecodeBounds = true
}
BitmapFactory.decodeFile(file.absolutePath, bounds)
val options = BitmapFactory.Options().apply {
inSampleSize = calculateBitmapSampleSize(bounds)
}
return BitmapFactory.decodeFile(file.absolutePath, options)
}
private fun calculateBitmapSampleSize(options: BitmapFactory.Options): Int {
val width = options.outWidth
val height = options.outHeight
if (width <= 0 || height <= 0) return 1
var sampleSize = 1
while ((width / sampleSize) > EPUB_COVER_MAX_DIMENSION || (height / sampleSize) > EPUB_COVER_MAX_DIMENSION) {
sampleSize *= 2
}
return sampleSize
}
private fun isEssentialFile(fileName: String, parseContent: Boolean): Boolean {
val lowerName = fileName.lowercase()
if (lowerName.endsWith("container.xml") || lowerName.endsWith(".opf")) {
return true
}
if (!parseContent) {
return false
}
return lowerName.endsWith(".xml") ||
lowerName.endsWith(".ncx") ||
lowerName.endsWith(".html") ||
lowerName.endsWith(".xhtml") ||
lowerName.endsWith(".htm") ||
lowerName.endsWith(".css")
}
}