Paging 3 with Mutable Data

Paging 3 is an easy and efficient way to handle paging of data in Android apps. On the internet, there are many articles about the basic implementation of Paging 3 to your project and also about implementation with Jetpack Compose, so it will not be part of this topic.

We will take a look at one of the disadvantages of Paging 3 which is immutability of already loaded data.

Why do we want to mutate loaded data?

There could be a lot of reasons, but for our example let’s imagine a small application to browse characters from the Rick and Morty series. We have a screen with a list of characters with search, and a detail screen of a specific character with more information. Data from the server is paginated, of course. We implement pagination with Paging 3 and all works well, unless we decide that we want to add an option to like any character we want. Here comes the problem, because when a user clicks on the like button, we want to change the state of the button to “liked”, but the loaded list of characters is now immutable and we don’t want to reload the whole list, each time the user clicks the like button.

Easy solution?

If we search on the internet or implement the first solution we think of, we can store these changes directly in ViewModel and show the actual state based on these changes.

That will work, but with this implementation come some issues. First, we have to implement it in each ViewModel where we will have a list of characters. Second issue comes when we add an option to like/unlike a character from the detail screen. If we change the state of like on the detail screen and we go back to the list, the state of the like button will stay in the previous state. This is a problem we have to solve by a shared ViewModel, or Fragment Result, which is a way to hell. 

Better solution?

To solve issues from the previous solution, we need to store the actual state of a character like somewhere, where it should be shared for the whole app and this state we need to combine with paginated data from the server. That's exactly what we will do and to keep the architecture correct, we want to implement all this in the data layer. In the presentation we want to get only really up-to-date data and nothing more.

On GitHub you can find an example project with real implementation on public API. All codes in this article are from this project 

Character cache

First of all, we will create a class to store actual states of character mutable data.

data class CachedCharacterPersonalisation(  
  
   val isLiked: Boolean? = null,  
)

This data class will represent our cached info about a character. Let’s call it personalisation and make it easy to add other attributes than just like. 

Now we can create an interface for our cache.

interface CharacterCache {  
  
   fun getCharacterPersonalisationStream(): Flow, CachedCharacterPersonalisation>>  
  
   fun updateCharacterIsLiked(characterId: CharacterId, isLiked: Boolean)  
}

This will be our class to store actual states of character personalisation. It will be injected as a singleton to be shared through the whole app. Let’s implement it.

private val cachedCharacterPersonalisationStream = _MutableStateFlow_CachedCharacterPersonalisation>>(_emptyMap_())

override fun updateCharacterIsLiked(characterId: CharacterId, isLiked: Boolean) {  
   updateCharacterCache(  
       characterId = characterId,  
       updatePersonalisation = **{** cachedViventPersonalisation **->** cachedViventPersonalisation.copy(isLiked = isLiked) **}**,  
   )  
}  
  
  
private fun updateCharacterCache(  
   characterId: CharacterId,  
   updatePersonalisation: (CachedCharacterPersonalisation) -> CachedCharacterPersonalisation,  
) {  
   val characterPersonalisation = cachedCharacterPersonalisationStream.value[characterId] ?: CachedCharacterPersonalisation()  
   val updatedCharacterPersonalisation = updatePersonalisation(characterPersonalisation)  
   cachedCharacterPersonalisationStream.value += characterId _to_ updatedCharacterPersonalisation  
}

When a character is liked, the personalisation state will be changed and a new state will be emitted by StateFlow.

This cached personalisation data will be merged with loaded data from Pager. For this purpose we will create a class named CachedPager.

Abstract Cached Pager

If we want to do all the work on the data layer, we also have to move our Pager there. 

We will create an abstract class called BaseCachedPagerwhich will create our paging source, create a Paging 3 pager and merge loaded data with our cache.

abstract class BaseCachedPager<DataType : Any, IdKey : Any, Personalisation : Any>(  
   coroutineScope: CoroutineScope,  
) {  
  
   private var pagingSource: DefaultPagingSource<DataType>? = null  
   val pagingDataStream: FlowDataType>> by _lazy_ {  
       Pager(PagingConfig(pageSize)) { createPagingSource()._apply_ { pagingSource = this } }  
           .flow  
           ._cachedIn_(coroutineScope)  
           ._combine_(getCachedInfoStream()) { pagingData, cachedPersonalisation ->  
               pagingData._map_ { item ->  
                   mergeWithCache(item, cachedPersonalisation)  
               }  
           }  
   }

We need to provide a coroutine scope because calling cachedIn is required to allow calling submitData on the same instance of PagingData emitted by Pager.

abstract fun getCachedInfoStream(): FlowIdKey, Personalisation>>  
  
abstract fun mergeWithCache(item: DataType, cachedInfo: Map<IdKey, Personalisation>): DataType  
  
abstract fun createPagingSource(): DefaultPagingSource <DataType>

These three functions will be implemented by an implementation of CachedPager for a specific class, in our case for Characters.

fun refresh() {  
   pagingSource?.invalidate()  
}

For refreshing data, we will use invalidating paging sources, which will cause the recreation of a new paging source and reloading all data.

For calling refresh from the presentation layer we will create a PagingHandle which will wrap our cached pager and allow call operations we want from the ViewModel.

class PagingHandleImpl<T : Any>(  
   cachedPager: BaseCachedPager<T, *, *>,  
) : PagingHandle<T> {  
  
   override val getPagingDataStream: () -> FlowT>> = **{** cachedPager.pagingDataStream **}**  
  
   override val refresh: () -> Unit = **{** cachedPager.refresh() **}**  
}  

For easier usage, we can create a function in AbstractCachedPager to create PagingHandle.

 open fun createPagingHandle(): PagingHandle<DataType> {  
  
   return PagingHandleImpl(this)  
}

We will keep it open to be able to have it overridden when we will create extended PagingHandles with more functions like for changing query for paginated search.

Specific Cached Pager

For our character we will create a specific implementation of AbstractCachedPager.

class CachedCharacterPager(  
   private val characterCache: CharacterCache,  
   coroutineScope: CoroutineScope,  
   private val pagingFactory: () -> DefaultPagingSource,  
) : BaseCachedPager, CharacterId, CachedCharacterPersonalisation>(coroutineScope) {  
  
   override fun getCachedInfoStream(): Flow, CachedCharacterPersonalisation>> {  
       return characterCache.getCharacterPersonalisationStream()  
   }  
  
   override fun mergeWithCache(item: Character, cachedInfo: Map, CachedCharacterPersonalisation>): Character {  
       return item._resolveCachedPersonalisation_(cachedInfo)  
   }  
  
   override fun createPagingSource(): DefaultPagingSource {  
       return pagingFactory()  
   }  
}

Lastly we need to implement a merging function for character and cache.

fun Character.resolveCachedPersonalisation(cache: Map, CachedCharacterPersonalisation>): Character {  
   val cachedPersonalisation = cache[id]  
   return if (cachedPersonalisation != null) {  
       copy(personalisation = cachedPersonalisation._mergeWithPersonalisation_(personalisation))  
   } else {  
       this  
   }  
}  
  
fun CachedCharacterPersonalisation.mergeWithPersonalisation(personalisation: CharacterPersonalisation): CharacterPersonalisation {  
   return CharacterPersonalisation(  
       isLiked = isLiked ?: personalisation.isLiked  
   )  
}

If there is no cached data for the current character, the merging function returns the original object. With this implementation, it is easy to add more personalisation attributes.

Implementation in repository

The place where we'll create our CachedPager and interact with the cache is the repository. Let’s create a function to get our PagingHandle with paginated data.

override fun getCharacterStream(coroutineScope: CoroutineScope): PagingHandle {  
   return CachedCharacterPager(characterCache, coroutineScope) **{**  
       DefaultPagingSource **{** pagingRequest **->**  
           remoteDataSource.getCharacters(pagingRequest)  
       **}**  
   **}**.createPagingHandle()  
}

We will create our CatchedPager with a PagingSource where we call the API for the actual page and return the PagingHandle.

Repository will store changes of likes to the API and also to our cache.

override suspend fun setCharacterIsLiked(characterId: CharacterId, isLiked: Boolean) {  
   characterCache.updateCharacterIsLiked(characterId, isLiked)  
   try {  
       remoteDataSource.setCharacterIsLiked(characterId, isLiked)  
   } catch (e: Exception) {  
       characterCache.updateCharacterIsLiked(characterId, !isLiked)  
       throw e  
   }  
}

We want to simulate the immediate interaction of a like button. We store the new state of the button, then call the API endpoint and in case of error we will return the original state.

What changes in the presentation layer?

The only differences in our new implementation are in the ViewModel.

init {  
   val charactersPagingHandle = characterRepository.getCharacterStream(_viewModelScope_)  
   setState **{** copy(charactersPagingHandle = charactersPagingHandle) **}**  
}

If we want to load data (in our case in the init function), we just call the repository function, and store the PagingHandle to ScreenState.

For refresh, we just easily call refresh on the PagingHandle.

fun refresh(){  
   screenState.charactersPagingHandle?.refresh?.invoke()  
}

Potential extensions

This solution can be easily extended for many use cases. The most useful extension should be to implement paginated search. We can extend CatchedPager for actual query and use it for creating PagingSource and after change just call our refresh function. To call a change of query from ViewModel we can also extend PagingHandle. You can fing the implementation of this extensionin the example project.

We can add more information to our cached object, like statistics (number of visits, likes, followers..) which can be updated in our cache each time data is received from the server. That will keep up-to-date data through the whole application.

We can add more functions to our Cached pager, such as filtering which will allow us to filter items based on user actions like, for example, disliking items.

This solution is targeted to cases when all personalisation data is stored primarily on a server. If you use a database for storing this data, you can use it instead of a shared cache. In our case, up-to-date data is obtained from the server when we reload the app, so our cached data can be cleared on app reload.

Conclusion

Paging 3 is an easy to use paging library which, with the implementation of CatchedPager, is able to easily work with mutable data. Also now up-to-date personalisation data is shared through the whole app and we can keep all data up-to-date without reloading. The implementation is prepared for easy extensions of data and functionality.